Abstract
This paper investigates the relations between theorems of the alternative and the minimum norm duality theorem. A typical theorem of the alternative is associated with two systems of linear inequalities and/or equalities, a primal system and a dual one, asserting that either the primal system has a solution, or the dual system has a solution, but never both. On the other hand, the minimum norm duality theorem says that the minimum distance from a given point z to a convex set \(\mathbb{K}\) is equal to the maximum of the distances from z to the hyperplanes separating z and \(\mathbb{K}\). We consider the theorems of Farkas, Gale, Gordan, and Motzkin, as well as new theorems that characterize the optimality conditions of discrete l 1-approximation problems and multifacility location problems. It is shown that, with proper choices of \(\mathbb{K}\), each of these theorems can be recast as a pair of dual problems: a primal steepest descent problem that resembles the original primal system, and a dual least–norm problem that resembles the original dual system. The norm that defines the least-norm problem is the dual norm with respect to that which defines the steepest descent problem. Moreover, let y solve the least norm problem and let r denote the corresponding residual vector. If r=0, which means that z ∈ \(\mathbb{K}\), then y solves the dual system. Otherwise, when r≠0 and z ∉ \(\mathbb{K}\), any dual vector of r solves both the steepest descent problem and the primal system. In other words, let x solve the steepest descent problem; then, r and x are aligned. These results hold for any norm on \(\mathbb{R}^n \). If the norm is smooth and strictly convex, then there are explicit rules for retrieving x from r and vice versa.
Similar content being viewed by others
References
MANGASARIAN, O. L., Nonlinear Programming, McGraw-Hill, New York, New York, 1969.
DAX, A., The Relationship between Theorems of the Alternative, Least Norm Problems, Steepest Descent Directions, and Degeneracy: A Review, Annals of Operations Research, Vol. 46, pp. 11–60, 1993.
NIRENBERG, L., Functional Analysis, Lectures Given in 1960–1961, Notes by Lesley Sibner, New York University, 1961.
LUENBERGER, D. G., Optimization by Vector Space Methods, Wiley, New York, New York, 1969.
FARKAS, J., Über die Theorie der Einfachen Ungleichungen, Journal für Reine und Angewandte Mathematik, Vol. 124, pp. 1–24, 1902.
GALE, D., The Theory of Linear Economic Models, McGraw-Hill, New York, New York, 1960.
GORDAN, P., Über die Auflösungen Linearer Gleichungen mit Reelen Coefficienten, Mathematische Annalen, Vol. 6, pp. 23–28, 1873.
MOTZKIN, T. S., Beiträge zur Theorie der Linearen Ungleichungen, Inaugural Dissertation, Basel, Switzerland, 1936.
DAX, A., A New Theorem of the Alternative, Mathematical Programming, Vol. 47, pp. 297–299, 1990.
MCLINDEN, L., Duality Theorems and Theorems of the Alternative, Proceedings of the American Mathematical Society, Vol. 53, pp. 172–175, 1975.
DAX, A., A Further Look at Theorems of the Alternative, Technical Report, Hydrological Service, Jerusalem, Israel, 1994.
HOUSEHOLDER, A. S., The Theory of Matrices in Numerical Analysis, Blaisdell Publishing Company, New York, New York, 1964.
STOER, J., and WITZGALL, C., Convexity and Optimization in Finite Dimensions, Part 1, Springer, Berlin, Germany, 1970.
ROBERTS, A. W., and VARBERG, D. E., Convex Functions, Academic Press, New York, New York, 1973.
BONNESEN, T., and FENCHEL, W., Theory of Convex Bodies, BCS Associates, Moscow, Russia, 1987.
SREEDHARAN, V. P., Solutions of Overdetermined Linear Equations Which Minimize Error in an Abstract Norm, Numerische Mathematik, Vol. 13, pp. 146–151, 1969.
CIARLET, P. G., Introduction to Numerical Linear Algebra and Optimization, Cambridge University Press, Cambridge, England, 1989.
OSBORNE, M. R., Finite Algorithms in Optimization and Data Analysis, John Wiley and Sons, Chichester, England, 1985.
DAX, A., An Elementary Proof of the Farkas' Lemma, SIAM Review, Vol. 39, 1987.
SREEDHARAN, V. P., An Algorithm for Nonnegative Norm Minimal Solutions, Numerical Functional Analysis and Optimization, Vol. 9, pp. 193–232, 1987.
DAX, A., Linear Programming via Least Squares, Linear Algebra and Its Applications, Vol. 111, pp. 313–324, 1988.
DAX, A., A Note on Minimum Norm Solutions, Journal of Optimization Theory and Applications, Vol. 76, pp. 183–193, 1993.
DAX, A., Methods for Calculating l p -Minimum Norm Solutions of Consistent Linear Systems, Journal of Optimization Theory and Applications, Vol. 83, pp. 333–353, 1993.
DAX, A., An Extended Kaczmarz Method for l p -Minimum Norm Solutions, Proceedings of the 4th International Symposium on Computerized Tomography, Novosibirsk, Russia, Edited by M. M. Lavrentev, VSP, Amsterdam, Netherlands, pp. 101–116, 1995.
BARTELS, R. H., CONN, A. R., and CHARALAMBOUS, C., On Cline's Direct Method for Solving Overdetermined Linear Systems in the l ∞ -Sense, SIAM Journal on Numerical Analysis, Vol. 15, pp. 255–270, 1978.
DAX, A., The Minimax Solution of Linear Equations Subject to Linear Constraints, IMA Journal of Numerical Analysis, Vol. 9, pp. 95–109, 1989.
DAX, A., Minimizing a Polyhedral Convex Function Subject to Linear Constraints, Technical Report, Hydrological Service, Jerusalem, Israel, 1987.
DEMJANOV, V. F., Algorithms for Some Minimax Problems, Journal of Computer and System Sciences, Vol. 2, pp. 342–380, 1968.
BERTSEKAS, D. P., and MITTER, S. K., A Descent Numerical Method for Optimization Problems with Nondifferentiable Cost Functionals, SIAM Journal on Control, Vol. 11, pp. 637–652, 1973.
WOLFE, P., A Method of Conjugate Subgradients for Minimizing Nondifferentiable Functions, Mathematical Programming Study, Vol. 3, pp. 145–173, 1975.
BUSOVACA, S., Handling Degeneracy in a Nonlinear l 1 -Algorithm, Technical Report 85–34, Computer Science Department, University of Waterloo, Waterloo, Ontario, Canada, 1985.
DAX, A., The l 1 -Solution of Linear Equations Subject to Linear Constraints, SIAM Journal on Scientific and Statistical Computing, Vol. 10, pp. 328–340, 1989.
DAX, A., On Regularized Least Norm Problems, SIAM Journal on Optimization, Vol. 2, pp. 602–618, 1992.
BARTELS, R. H., CONN, A. R., and SINCLAIR, J. W., Minimization Techniques for Piecewise Differentiable Functions: The l 1 -Solution to an Overdetermined Linear System, SIAM Journal on Numerical Analysis, Vol. 15, pp. 224–241, 1978.
BARTELS, R. H., and CONN, A. R., Linearly Constrained Discrete l 1 -Problems, ACM Transactions on Mathematical Software, Vol. 6, pp. 594–608, 1980.
KAPLAN, W., and YANG, W. H., Duality Theorem for a Generalized Fermat-Weber Problem, Technical Report, Mathematics Department, University of Michigan, Ann Arbor, Michigan, 1995.
PRITSKER, A. A. B., and GHARE, P. M., Locating New Facilities with Respect to Existing Facilities, Transactions of the American Institute of Industrial Engineers, Vol. 2, pp. 290–297, 1970.
JUEL, H., and LOVE, R. F., An Efficient Computational Procedure for Solving the Multifacility Rectilinear Facilities Location Problem, Operational Research Quarterly, Vol. 27, pp. 697-703, 1976.
CALAMAI, P., and CHARALAMBOUS, C., Solving Multifacility Location Problems Involving Euclidean Distances, Naval Research Logistics Quarterly, Vol. 27, pp. 609-620, 1980.
JUEL, H., A Note on Solving Multifacility Location Problems Involving Euclidean Distance, Naval Research Logistics Quarterly, Vol. 29, pp. 179-180, 1982.
OVERTON, M. L., A Quadratically Convergent Method for Minimizing a Sum of Euclidean Norms, Mathematical Programming, Vol. 27, pp. 34-63, 1983.
DAX, A., A Note on Optimality Conditions for the Euclidean Multifacility Location Problem, Mathematical Programming, Vol. 36, pp. 72-80, 1986.
DAX, A., An Efficient Algorithm for Solving the Rectilinear Multifacility Location Problem, IMA Journal of Numerical Analysis, Vol. 6, pp. 343-355, 1986.
CALAMAI, P. H., and CONN, A. R., A Projected Newton Method for l p -Norm Location Problems, Mathematical Programming, Vol. 38, pp. 75-109, 1987.
Rights and permissions
About this article
Cite this article
Dax, A., Sreedharan, V.P. Theorems of the Alternative and Duality. Journal of Optimization Theory and Applications 94, 561–590 (1997). https://doi.org/10.1023/A:1022644832111
Issue Date:
DOI: https://doi.org/10.1023/A:1022644832111