Top Qs
Timeline
Chat
Perspective
Hilbert projection theorem
On closed convex subsets in Hilbert space From Wikipedia, the free encyclopedia
Remove ads
Remove ads
In mathematics, the Hilbert projection theorem is a famous result of convex analysis that says that for every vector in a Hilbert space and every nonempty closed convex there exists a unique vector for which is minimized over the vectors ; that is, such that for every
This article relies largely or entirely on a single source. (February 2020) |
Remove ads
Finite dimensional case
Summarize
Perspective
Some intuition for the theorem can be obtained by considering the first order condition of the optimization problem.
Consider a finite dimensional real Hilbert space with a subspace and a point If is a minimizer or minimum point of the function defined by (which is the same as the minimum point of ), then derivative must be zero at
In matrix derivative notation[1] Since is a vector in that represents an arbitrary tangent direction, it follows that must be orthogonal to every vector in
Remove ads
Statement
Hilbert projection theorem—For every vector in a Hilbert space and every nonempty closed convex there exists a unique vector for which is equal to
If the closed subset is also a vector subspace of then this minimizer is the unique element in such that is orthogonal to
Detailed elementary proof
Proof that a minimum point exists
Let be the distance between and a sequence in such that the distance squared between and is less than or equal to Let and be two integers, then the following equalities are true: and Therefore (This equation is the same as the formula for the length of a median in a triangle with sides of length and where specifically, the triangle's vertices are ).
By giving an upper bound to the first two terms of the equality and by noticing that the midpoint of and belong to and has therefore a distance greater than or equal to from it follows that:
The last inequality proves that is a Cauchy sequence. Since is complete, the sequence is therefore convergent to a point whose distance from is minimal.
Proof that is unique
Let and be two minimum points. Then:
Since belongs to we have and therefore
Hence which proves uniqueness.
Proof of characterization of minimum point when is a closed vector subspace
Assume that is a closed vector subspace of It must be shown the minimizer is the unique element in such that for every
Proof that the condition is sufficient: Let be such that for all If then and so which implies that Because was arbitrary, this proves that and so is a minimum point.
Proof that the condition is necessary: Let be the minimum point. Let and Because the minimality of guarantees that Thus is always non-negative and must be a real number. If then the map has a minimum at and moreover, which is a contradiction. Thus
Proof by reduction to a special case
It suffices to prove the theorem in the case of because the general case follows from the statement below by replacing with
Hilbert projection theorem (case )[2]—For every nonempty closed convex subset of a Hilbert space there exists a unique vector such that
Furthermore, letting if is any sequence in such that in [note 1] then in
Proof
Let be as described in this theorem and let This theorem will follow from the following lemmas.
Lemma 1—If is any sequence in such that in then there exists some such that in Furthermore,
Lemma 2—A sequence satisfying the hypotheses of Lemma 1 exists.
Lemma 2 and Lemma 1 together prove that there exists some such that Lemma 1 can be used to prove uniqueness as follows. Suppose is such that and denote the sequence by so that the subsequence of even indices is the constant sequence while the subsequence of odd indices is the constant sequence Because for every in which shows that the sequence satisfies the hypotheses of Lemma 1. Lemma 1 guarantees the existence of some such that in Because converges to so do all of its subsequences. In particular, the subsequence converges to which implies that (because limits in are unique and this constant subsequence also converges to ). Similarly, because the subsequence converges to both and Thus which proves the theorem.
Remove ads
Consequences
Proposition—If is a closed vector subspace of a Hilbert space then[note 3]
Remove ads
Properties
Summarize
Perspective
Expression as a global minimum
The statement and conclusion of the Hilbert projection theorem can be expressed in terms of global minimums of the followings functions. Their notation will also be used to simplify certain statements.
Given a non-empty subset and some define a function A global minimum point of if one exists, is any point in such that in which case is equal to the global minimum value of the function which is:
Effects of translations and scalings
When this global minimum point exists and is unique then denote it by explicitly, the defining properties of (if it exists) are: The Hilbert projection theorem guarantees that this unique minimum point exists whenever is a non-empty closed and convex subset of a Hilbert space. However, such a minimum point can also exist in non-convex or non-closed subsets as well; for instance, just as long is is non-empty, if then
If is a non-empty subset, is any scalar, and are any vectors then which implies:
Examples
The following counter-example demonstrates a continuous linear isomorphism for which Endow with the dot product, let and for every real let be the line of slope through the origin, where it is readily verified that Pick a real number and define by (so this map scales the coordinate by while leaving the coordinate unchanged). Then is an invertible continuous linear operator that satisfies and so that and Consequently, if with and if then
Remove ads
See also
- Orthogonal complement – Concept in linear algebra
- Orthogonal projection – Idempotent linear transformation from a vector space to itself
- Orthogonality principle – Condition for optimality of Bayesian estimator
- Riesz representation theorem – Theorem about the dual of a Hilbert space
Notes
- Because the norm is continuous, if converges in then necessarily converges in But in general, the converse is not guaranteed. However, under this theorem's hypotheses, knowing that in is sufficient to conclude that converges in
- Explicitly, this means that given any there exists some integer such that "the quantity" is whenever Here, "the quantity" refers to the inequality's right hand side and later in the proof, "the quantity" will also refer to and then By definition of "Cauchy sequence," is Cauchy in if and only if "the quantity" satisfies this aforementioned condition.
- Technically, means that the addition map defined by is a surjective linear isomorphism and homeomorphism. See the article on complemented subspaces for more details.
Remove ads
References
Bibliography
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads