Inner Products

Inner products generalize the idea of the dot product in to more general vector spaces. They are useful in defining angles and lengths of vectors.

Inner Product

Let be a vector space on . A function is called an inner product on if it satisfies the following properties:

1. Positivity: for all , and .
2. Symmetry: for all .
3. Sequilinearity: for all and there holds
4. Induced Norm: inner product induces a norm .

A vector space endowed with an inner product is called an inner product space. Sometimes, an inner product on a complex vector space is called a Hermitian form.

We shall verify that the induced norm is indeed a norm. Since , is always real. So the norm is real as expected. The triangle inequality follows from the Cauchy-Schwarz inequality, which we will prove later. The homogeneity property follows from the linearity of the inner product. The positivity property follows from the definition of the inner product.

e.g. is an inner product space under the dot product.

Orthogonality

Two vectors in an inner product space , is said to be orthogonal if .

Pythagorean Theorem

Let are two orthogonal vectors in an inner product space , if and only if

Proof as .

Cauchy-Schwarz Inequality

Let be an inner product space with induced norm . Then we have with equality iff is a scalar multiplication of .

Proof If , then both sides are zero, so we can assume . Then we can write By denoting , we observe that , that is, they are orthogonal. Thus, by Pythagorean theorem, we have Multiplying both sides of this inequality by and then taking square roots gives the desired inequality. And the equality holds iff , i.e. is a scalar multiple of .

Jordan-von Neumann Theorem

A norm on a vector space arises from an inner product if and only if the parallelogram law holds:If so, we have the induced inner product:

Orthogonal Transformation

An orthogonal transformation is a linear transformation on a inner product space that preserves the inner product, i.e. for all .

Proposition

In finite-dimensional vector spaces, the matrix representation (with respect to an orthonormal basis) of an orthogonal transformation is an orthogonal matrix.

Adjoints

Theorem

A bounded linear map on the inner product space is a projection if and only if and it is self-adjoint.

Proof If is a projection, then clearly , and for all , we have so is self-adjoint. Conversely, if is self-adjoint and , then consider , for which any element is preserved by . We claim that . For all and , there holds thus and . Conversely, for any , we want to show that . In fact, fix some , we have thus . Moreover, since , we have , thus . So has to be zero, , and we have . Note that must be closed, thus , so we have for all , which means is a projection onto .

Corollary

Suppose and are projections. Then is a projection if and only if and commute.

Proof If is a projection, then Conversely, suppose and commutes, then Moreover, we have $$\langle (P_{1}\circ P_{2})(u),v\rangle = \langle P_{2}(u),P_{1}(v)\rangle=\langle u,(P_{2}\circ P_{1})(v)\rangle=\langle u,(P_{1}\circ P_{2})(v)\rangle.$$$\square$