Inner Product Spaces and Orthonormal Bases

The Cauchy–Schwarz inequality |<u,v>| <= ||u|| ||v|| is the cornerstone of inner product spaces. We prove it from the axioms, then develop the Gram–Schmidt process for constructing orthonormal bases, orthogonal complements, and orthogonal projections that give the closest point in a subspace.

Folio Official

March 1, 2026

1. The Definition of an Inner Product

Definition 1 (Inner product).

A map

⟨ \cdot, \cdot ⟩ : V \times V \to R

on a real vector space

V

is called an inner product if it satisfies the following three axioms:

Symmetry. $⟨ u, v ⟩ = ⟨ v, u ⟩$ for all $u, v \in V$ .
Linearity in the first argument. $⟨ a u + b v, w ⟩ = a ⟨ u, w ⟩ + b ⟨ v, w ⟩$ for all $a, b \in R$ and $u, v, w \in V$ .
Positive definiteness. $⟨ v, v ⟩ \geq 0$ for all $v \in V$ , with equality if and only if $v = 0$ .

A vector space equipped with an inner product is called an inner product space.

Definition 2 (Norm and orthogonality).

In an inner product space

V

, the norm (or length) of a vector

v

is defined by

∥ v ∥ = ⟨ v, v ⟩

. Two vectors

u, v

are said to be orthogonal if

⟨ u, v ⟩ = 0

; we write

u ⊥ v

Example 3.

The standard inner product on $R^{n}$ : $⟨ x, y ⟩ = \sum_{i = 1}^{n} x_{i} y_{i} = x^{T} y$ .
An inner product on $C [a, b]$ : $⟨ f, g ⟩ = \int_{a}^{b} f (x) g (x) d x$ .
An inner product on $M_{n} (R)$ : $⟨ A, B ⟩ = tr (A^{T} B)$ .

2. The Cauchy–Schwarz Inequality

Theorem 4 (Cauchy–Schwarz inequality).

For any vectors

u, v

in an inner product space

V

∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥ \cdot ∥ v ∥.

Equality holds if and only if

u

and

v

are linearly dependent.

Proof.

v = 0

, both sides vanish and the inequality is trivially satisfied. Assume

v \neq = 0

. For any

t \in R

, positive definiteness gives

0 \leq ∥ u - t v ∥^{2} = ⟨ u - t v, u - t v ⟩ = ∥ u ∥^{2} - 2 t ⟨ u, v ⟩ + t^{2} ∥ v ∥^{2} .

Substituting

t = ⟨ u, v ⟩ /∥ v ∥^{2}

yields

0 \leq ∥ u ∥^{2} - \frac{⟨ u , v ⟩ ^{2}}{∥ v ∥ ^{2}},

and rearranging gives

⟨ u, v ⟩^{2} \leq ∥ u ∥^{2} ∥ v ∥^{2}

. □

Theorem 5 (Triangle inequality).

For any vectors

u, v

in an inner product space,

∥ u + v ∥ \leq ∥ u ∥ + ∥ v ∥

Proof.

We compute

∥ u + v ∥^{2} = ∥ u ∥^{2} + 2 ⟨ u, v ⟩ + ∥ v ∥^{2} \leq ∥ u ∥^{2} + 2∥ u ∥∥ v ∥ + ∥ v ∥^{2} = (∥ u ∥ + ∥ v ∥)^{2},

where the inequality follows from

⟨ u, v ⟩ \leq ∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥∥ v ∥

(the Cauchy–Schwarz inequality). Since both sides are nonnegative, taking square roots gives the result. □

3. Orthonormal Bases

Definition 6 (Orthonormal system and orthonormal basis).

A set

{e_{1}, \dots, e_{n}}

is called an orthonormal system if

⟨ e_{i}, e_{j} ⟩ = δ_{ij} = {10 if i = j, if i \neq = j .

An orthonormal system that is also a basis is called an orthonormal basis (ONB).

Theorem 7.

Every orthonormal system is linearly independent.

Proof.

Suppose

\sum_{i} c_{i} e_{i} = 0

. Taking the inner product with

e_{k}

gives

c_{k} = ⟨ \sum_{i} c_{i} e_{i}, e_{k} ⟩ = ⟨ 0, e_{k} ⟩ = 0

. □

Theorem 8 (Fourier expansion).

{e_{1}, \dots, e_{n}}

is an orthonormal basis for

V

, then every

v \in V

admits the expansion

v = i = 1 \sum n ⟨ v, e_{i} ⟩ e_{i} .

In other words, the

i

-th coordinate of

v

with respect to the orthonormal basis is simply

⟨ v, e_{i} ⟩

Proof.

Since

{e_{1}, \dots, e_{n}}

is a basis, we may write

v = \sum_{i = 1}^{n} a_{i} e_{i}

uniquely. Taking the inner product of both sides with

e_{k}

gives

⟨ v, e_{k} ⟩ = \sum_{i = 1}^{n} a_{i} ⟨ e_{i}, e_{k} ⟩ = \sum_{i = 1}^{n} a_{i} δ_{ik} = a_{k}

. □

4. The Gram–Schmidt Process

Theorem 9 (Gram–Schmidt orthonormalization).

Let

{v_{1}, \dots, v_{n}}

be a linearly independent set in an inner product space

V

. The following procedure produces an orthonormal set

{e_{1}, \dots, e_{n}}

Set $u_{1} = v_{1}$ and $e_{1} = u_{1} /∥ u_{1} ∥$ .
For $k = 2, \dots, n$ : set $u_{k} = v_{k} - \sum_{j = 1}^{k - 1} ⟨ v_{k}, e_{j} ⟩ e_{j}$ and $e_{k} = u_{k} /∥ u_{k} ∥$ .

Moreover,

span {e_{1}, \dots, e_{k}} = span {v_{1}, \dots, v_{k}}

at each stage.

Proof.

We proceed by induction on

k

. The case

k = 1

is clear. Suppose

{e_{1}, \dots, e_{k - 1}}

is an orthonormal basis for

span {v_{1}, \dots, v_{k - 1}}

. Define

u_{k} = v_{k} - \sum_{j = 1}^{k - 1} ⟨ v_{k}, e_{j} ⟩ e_{j}

. For each

j \leq k - 1

, we have

⟨ u_{k}, e_{j} ⟩ = ⟨ v_{k}, e_{j} ⟩ - ⟨ v_{k}, e_{j} ⟩ = 0

. Since

v_{k}

is not in

span {v_{1}, \dots, v_{k - 1}}

(by linear independence), we have

u_{k} \neq = 0

. Setting

e_{k} = u_{k} /∥ u_{k} ∥

completes the inductive step, and clearly

span {e_{1}, \dots, e_{k}} = span {v_{1}, \dots, v_{k}}

. □

5. Orthogonal Complements

Definition 10 (Orthogonal complement).

Let

W

be a subspace of an inner product space

V

. The orthogonal complement of

W

W^{⊥} = {v \in V ∣ ⟨ v, w ⟩ = 0 for all w \in W} .

Theorem 11.

Let

V

be a finite-dimensional inner product space and

W

a subspace of

V

. Then:

$W^{⊥}$ is a subspace of $V$ .
$V = W \oplus W^{⊥}$ .
$dim W^{⊥} = dim V - dim W$ .
$(W^{⊥})^{⊥} = W$ .

Proof.

(1) For

u_{1}, u_{2} \in W^{⊥}

and scalars

a, b

, we have

⟨ a u_{1} + b u_{2}, w ⟩ = a ⟨ u_{1}, w ⟩ + b ⟨ u_{2}, w ⟩ = 0

for all

w \in W

, so

a u_{1} + b u_{2} \in W^{⊥}

(2) Apply the Gram–Schmidt process to obtain an orthonormal basis

{e_{1}, \dots, e_{k}}

for

W

. Given any

v \in V

, set

w = \sum_{i = 1}^{k} ⟨ v, e_{i} ⟩ e_{i} \in W

. Then for each

j \leq k

⟨ v - w, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - ⟨ v, e_{j} ⟩ = 0

, so

v - w \in W^{⊥}

. Thus

v = w + (v - w)

gives

V = W + W^{⊥}

. The intersection

W \cap W^{⊥} = {0}

is immediate: if

u \in W \cap W^{⊥}

, then

⟨ u, u ⟩ = 0

, hence

u = 0

(3) From the direct sum decomposition,

dim V = dim W + dim W^{⊥}

(4) The inclusion

W \subseteq (W^{⊥})^{⊥}

follows from the definition. By (3),

dim (W^{⊥})^{⊥} = dim V - dim W^{⊥} = dim W

, so

W = (W^{⊥})^{⊥}

. □

6. Orthogonal Projection

Definition 12 (Orthogonal projection).

Given the decomposition

V = W \oplus W^{⊥}

, every

v \in V

can be written uniquely as

v = w + w^{⊥}

with

w \in W

and

w^{⊥} \in W^{⊥}

. The vector

w

is called the orthogonal projection of

v

onto

W

, denoted

proj_{W} (v) = w

Theorem 13.

{e_{1}, \dots, e_{k}}

is an orthonormal basis for

W

, then

proj_{W} (v) = i = 1 \sum k ⟨ v, e_{i} ⟩ e_{i} .

Proof.

Set

w = \sum_{i = 1}^{k} ⟨ v, e_{i} ⟩ e_{i} \in W

. For each

j = 1, \dots, k

⟨ v - w, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - i = 1 \sum k ⟨ v, e_{i} ⟩ ⟨ e_{i}, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - ⟨ v, e_{j} ⟩ = 0,

v - w \in W^{⊥}

. The decomposition

v = w + (v - w)

is the direct-sum decomposition

V = W \oplus W^{⊥}

, whence

proj_{W} (v) = w

. □

Theorem 14 (Best approximation theorem).

The projection

proj_{W} (v)

is the closest point in

W

v

∥ v - proj_{W} (v) ∥ \leq ∥ v - w ∥ for all w \in W,

with equality if and only if

w = proj_{W} (v)

Proof.

Let

\overset{v}{^} = proj_{W} (v)

. For any

w \in W

∥ v - w ∥^{2} = ∥ v - \overset{v}{^} + \overset{v}{^} - w ∥^{2} = ∥ v - \overset{v}{^} ∥^{2} + ∥ \overset{v}{^} - w ∥^{2},

where the cross term vanishes because

v - \overset{v}{^} \in W^{⊥}

and

\overset{v}{^} - w \in W

are orthogonal. Since

∥ \overset{v}{^} - w ∥^{2} \geq 0

, we obtain

∥ v - w ∥^{2} \geq ∥ v - \overset{v}{^} ∥^{2}

, with equality only when

\overset{v}{^} = w

. □

Linear Algebra Algebra Textbook Inner Product Space Gram-Schmidt Orthogonal Complement

Folio Official

Mathematics "between the lines" — exploring the intuition textbooks leave out, written in LaTeX on Folio.

1 followers·107 articles

Inner Product Spaces and Orthonormal Bases

Folio Official

March 1, 2026

1. The Definition of an Inner Product

Definition 1 (Inner product).

A map

⟨ \cdot, \cdot ⟩ : V \times V \to R

on a real vector space

V

is called an inner product if it satisfies the following three axioms:

Symmetry. $⟨ u, v ⟩ = ⟨ v, u ⟩$ for all $u, v \in V$ .
Linearity in the first argument. $⟨ a u + b v, w ⟩ = a ⟨ u, w ⟩ + b ⟨ v, w ⟩$ for all $a, b \in R$ and $u, v, w \in V$ .
Positive definiteness. $⟨ v, v ⟩ \geq 0$ for all $v \in V$ , with equality if and only if $v = 0$ .

A vector space equipped with an inner product is called an inner product space.

Definition 2 (Norm and orthogonality).

In an inner product space

V

, the norm (or length) of a vector

v

is defined by

∥ v ∥ = ⟨ v, v ⟩

. Two vectors

u, v

are said to be orthogonal if

⟨ u, v ⟩ = 0

; we write

u ⊥ v

Example 3.

The standard inner product on $R^{n}$ : $⟨ x, y ⟩ = \sum_{i = 1}^{n} x_{i} y_{i} = x^{T} y$ .
An inner product on $C [a, b]$ : $⟨ f, g ⟩ = \int_{a}^{b} f (x) g (x) d x$ .
An inner product on $M_{n} (R)$ : $⟨ A, B ⟩ = tr (A^{T} B)$ .

2. The Cauchy–Schwarz Inequality

Theorem 4 (Cauchy–Schwarz inequality).

For any vectors

u, v

in an inner product space

V

∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥ \cdot ∥ v ∥.

Equality holds if and only if

u

and

v

are linearly dependent.

Proof.

v = 0

, both sides vanish and the inequality is trivially satisfied. Assume

v \neq = 0

. For any

t \in R

, positive definiteness gives

0 \leq ∥ u - t v ∥^{2} = ⟨ u - t v, u - t v ⟩ = ∥ u ∥^{2} - 2 t ⟨ u, v ⟩ + t^{2} ∥ v ∥^{2} .

Substituting

t = ⟨ u, v ⟩ /∥ v ∥^{2}

yields

0 \leq ∥ u ∥^{2} - \frac{⟨ u , v ⟩ ^{2}}{∥ v ∥ ^{2}},

and rearranging gives

⟨ u, v ⟩^{2} \leq ∥ u ∥^{2} ∥ v ∥^{2}

. □

Theorem 5 (Triangle inequality).

For any vectors

u, v

in an inner product space,

∥ u + v ∥ \leq ∥ u ∥ + ∥ v ∥

Proof.

We compute

∥ u + v ∥^{2} = ∥ u ∥^{2} + 2 ⟨ u, v ⟩ + ∥ v ∥^{2} \leq ∥ u ∥^{2} + 2∥ u ∥∥ v ∥ + ∥ v ∥^{2} = (∥ u ∥ + ∥ v ∥)^{2},

where the inequality follows from

⟨ u, v ⟩ \leq ∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥∥ v ∥

(the Cauchy–Schwarz inequality). Since both sides are nonnegative, taking square roots gives the result. □

3. Orthonormal Bases

Definition 6 (Orthonormal system and orthonormal basis).

A set

{e_{1}, \dots, e_{n}}

is called an orthonormal system if

⟨ e_{i}, e_{j} ⟩ = δ_{ij} = {10 if i = j, if i \neq = j .

An orthonormal system that is also a basis is called an orthonormal basis (ONB).

Theorem 7.

Every orthonormal system is linearly independent.

Proof.

Suppose

\sum_{i} c_{i} e_{i} = 0

. Taking the inner product with

e_{k}

gives

c_{k} = ⟨ \sum_{i} c_{i} e_{i}, e_{k} ⟩ = ⟨ 0, e_{k} ⟩ = 0

. □

Theorem 8 (Fourier expansion).

{e_{1}, \dots, e_{n}}

is an orthonormal basis for

V

, then every

v \in V

admits the expansion

v = i = 1 \sum n ⟨ v, e_{i} ⟩ e_{i} .

In other words, the

i

-th coordinate of

v

with respect to the orthonormal basis is simply

⟨ v, e_{i} ⟩

Proof.

Since

{e_{1}, \dots, e_{n}}

is a basis, we may write

v = \sum_{i = 1}^{n} a_{i} e_{i}

uniquely. Taking the inner product of both sides with

e_{k}

gives

⟨ v, e_{k} ⟩ = \sum_{i = 1}^{n} a_{i} ⟨ e_{i}, e_{k} ⟩ = \sum_{i = 1}^{n} a_{i} δ_{ik} = a_{k}

. □

4. The Gram–Schmidt Process

Theorem 9 (Gram–Schmidt orthonormalization).

Let

{v_{1}, \dots, v_{n}}

be a linearly independent set in an inner product space

V

. The following procedure produces an orthonormal set

{e_{1}, \dots, e_{n}}

Set $u_{1} = v_{1}$ and $e_{1} = u_{1} /∥ u_{1} ∥$ .
For $k = 2, \dots, n$ : set $u_{k} = v_{k} - \sum_{j = 1}^{k - 1} ⟨ v_{k}, e_{j} ⟩ e_{j}$ and $e_{k} = u_{k} /∥ u_{k} ∥$ .

Moreover,

span {e_{1}, \dots, e_{k}} = span {v_{1}, \dots, v_{k}}

at each stage.

Proof.

We proceed by induction on

k

. The case

k = 1

is clear. Suppose

{e_{1}, \dots, e_{k - 1}}

is an orthonormal basis for

span {v_{1}, \dots, v_{k - 1}}

. Define

u_{k} = v_{k} - \sum_{j = 1}^{k - 1} ⟨ v_{k}, e_{j} ⟩ e_{j}

. For each

j \leq k - 1

, we have

⟨ u_{k}, e_{j} ⟩ = ⟨ v_{k}, e_{j} ⟩ - ⟨ v_{k}, e_{j} ⟩ = 0

. Since

v_{k}

is not in

span {v_{1}, \dots, v_{k - 1}}

(by linear independence), we have

u_{k} \neq = 0

. Setting

e_{k} = u_{k} /∥ u_{k} ∥

completes the inductive step, and clearly

span {e_{1}, \dots, e_{k}} = span {v_{1}, \dots, v_{k}}

. □

5. Orthogonal Complements

Definition 10 (Orthogonal complement).

Let

W

be a subspace of an inner product space

V

. The orthogonal complement of

W

W^{⊥} = {v \in V ∣ ⟨ v, w ⟩ = 0 for all w \in W} .

Theorem 11.

Let

V

be a finite-dimensional inner product space and

W

a subspace of

V

. Then:

$W^{⊥}$ is a subspace of $V$ .
$V = W \oplus W^{⊥}$ .
$dim W^{⊥} = dim V - dim W$ .
$(W^{⊥})^{⊥} = W$ .

Proof.

(1) For

u_{1}, u_{2} \in W^{⊥}

and scalars

a, b

, we have

⟨ a u_{1} + b u_{2}, w ⟩ = a ⟨ u_{1}, w ⟩ + b ⟨ u_{2}, w ⟩ = 0

for all

w \in W

, so

a u_{1} + b u_{2} \in W^{⊥}

(2) Apply the Gram–Schmidt process to obtain an orthonormal basis

{e_{1}, \dots, e_{k}}

for

W

. Given any

v \in V

, set

w = \sum_{i = 1}^{k} ⟨ v, e_{i} ⟩ e_{i} \in W

. Then for each

j \leq k

⟨ v - w, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - ⟨ v, e_{j} ⟩ = 0

, so

v - w \in W^{⊥}

. Thus

v = w + (v - w)

gives

V = W + W^{⊥}

. The intersection

W \cap W^{⊥} = {0}

is immediate: if

u \in W \cap W^{⊥}

, then

⟨ u, u ⟩ = 0

, hence

u = 0

(3) From the direct sum decomposition,

dim V = dim W + dim W^{⊥}

(4) The inclusion

W \subseteq (W^{⊥})^{⊥}

follows from the definition. By (3),

dim (W^{⊥})^{⊥} = dim V - dim W^{⊥} = dim W

, so

W = (W^{⊥})^{⊥}

. □

6. Orthogonal Projection

Definition 12 (Orthogonal projection).

Given the decomposition

V = W \oplus W^{⊥}

, every

v \in V

can be written uniquely as

v = w + w^{⊥}

with

w \in W

and

w^{⊥} \in W^{⊥}

. The vector

w

is called the orthogonal projection of

v

onto

W

, denoted

proj_{W} (v) = w

Theorem 13.

{e_{1}, \dots, e_{k}}

is an orthonormal basis for

W

, then

proj_{W} (v) = i = 1 \sum k ⟨ v, e_{i} ⟩ e_{i} .

Proof.

Set

w = \sum_{i = 1}^{k} ⟨ v, e_{i} ⟩ e_{i} \in W

. For each

j = 1, \dots, k

⟨ v - w, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - i = 1 \sum k ⟨ v, e_{i} ⟩ ⟨ e_{i}, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - ⟨ v, e_{j} ⟩ = 0,

v - w \in W^{⊥}

. The decomposition

v = w + (v - w)

is the direct-sum decomposition

V = W \oplus W^{⊥}

, whence

proj_{W} (v) = w

. □

Theorem 14 (Best approximation theorem).

The projection

proj_{W} (v)

is the closest point in

W

v

∥ v - proj_{W} (v) ∥ \leq ∥ v - w ∥ for all w \in W,

with equality if and only if

w = proj_{W} (v)

Proof.

Let

\overset{v}{^} = proj_{W} (v)

. For any

w \in W

∥ v - w ∥^{2} = ∥ v - \overset{v}{^} + \overset{v}{^} - w ∥^{2} = ∥ v - \overset{v}{^} ∥^{2} + ∥ \overset{v}{^} - w ∥^{2},

where the cross term vanishes because

v - \overset{v}{^} \in W^{⊥}

and

\overset{v}{^} - w \in W

are orthogonal. Since

∥ \overset{v}{^} - w ∥^{2} \geq 0

, we obtain

∥ v - w ∥^{2} \geq ∥ v - \overset{v}{^} ∥^{2}

, with equality only when

\overset{v}{^} = w

. □

Linear Algebra Algebra Textbook Inner Product Space Gram-Schmidt Orthogonal Complement

Folio Official

Mathematics "between the lines" — exploring the intuition textbooks leave out, written in LaTeX on Folio.

1 followers·107 articles

Inner Product Spaces and Orthonormal Bases

1. The Definition of an Inner Product

2. The Cauchy–Schwarz Inequality

3. Orthonormal Bases

4. The Gram–Schmidt Process

5. Orthogonal Complements

6. Orthogonal Projection

Share your expertise with the world

More from Folio Official

Matrices and Representation of Linear Maps

The Determinant: A Scalar Invariant of Square Matrices

Jordan Normal Form: Beyond Diagonalization

The Singular Value Decomposition: Structure of Arbitrary Matrices

Inner Product Spaces and Orthonormal Bases

1. The Definition of an Inner Product

2. The Cauchy–Schwarz Inequality

3. Orthonormal Bases

4. The Gram–Schmidt Process

5. Orthogonal Complements

6. Orthogonal Projection

Share your expertise with the world

More from Folio Official

Matrices and Representation of Linear Maps

The Determinant: A Scalar Invariant of Square Matrices

Jordan Normal Form: Beyond Diagonalization

The Singular Value Decomposition: Structure of Arbitrary Matrices