Inner Product Spaces and Orthonormal Bases

Starting from the axiomatic definition of an inner product, we derive the Cauchy--Schwarz inequality and the triangle inequality. We then develop the Gram--Schmidt orthonormalization process, the theory of orthogonal complements, orthogonal projections, and the best-approximation theorem.

Folio Official

March 1, 2026

1 The Definition of an Inner Product

Definition 1 (Inner product).

A map

⟨ \cdot, \cdot ⟩ : V \times V \to R

on a real vector space

V

is called an inner product if it satisfies the following three axioms:

Symmetry. $⟨ u, v ⟩ = ⟨ v, u ⟩$ for all $u, v \in V$ .
Linearity in the first argument. $⟨ a u + b v, w ⟩ = a ⟨ u, w ⟩ + b ⟨ v, w ⟩$ for all $a, b \in R$ and $u, v, w \in V$ .
Positive definiteness. $⟨ v, v ⟩ \geq 0$ for all $v \in V$ , with equality if and only if $v = 0$ .

A vector space equipped with an inner product is called an inner product space.

Definition 2 (Norm and orthogonality).

In an inner product space

V

, the norm (or length) of a vector

v

is defined by

∥ v ∥ = ⟨ v, v ⟩

. Two vectors

u, v

are said to be orthogonal if

⟨ u, v ⟩ = 0

; we write

u ⊥ v

Example 3.

The standard inner product on $R^{n}$ : $⟨ x, y ⟩ = \sum_{i = 1}^{n} x_{i} y_{i} = x^{T} y$ .
An inner product on $C [a, b]$ : $⟨ f, g ⟩ = \int_{a}^{b} f (x) g (x) d x$ .
An inner product on $M_{n} (R)$ : $⟨ A, B ⟩ = tr (A^{T} B)$ .

2 The Cauchy–Schwarz Inequality

Theorem 4 (Cauchy–Schwarz inequality).

For any vectors

u, v

in an inner product space

V

∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥ \cdot ∥ v ∥.

Equality holds if and only if

u

and

v

are linearly dependent.

Proof.

v = 0

, both sides vanish and the inequality is trivially satisfied. Assume

v \neq = 0

. For any

t \in R

, positive definiteness gives

0 \leq ∥ u - t v ∥^{2} = ⟨ u - t v, u - t v ⟩ = ∥ u ∥^{2} - 2 t ⟨ u, v ⟩ + t^{2} ∥ v ∥^{2} .

Substituting

t = ⟨ u, v ⟩ /∥ v ∥^{2}

yields

0 \leq ∥ u ∥^{2} - \frac{⟨ u , v ⟩ ^{2}}{∥ v ∥ ^{2}},

and rearranging gives

⟨ u, v ⟩^{2} \leq ∥ u ∥^{2} ∥ v ∥^{2}

. □

Theorem 5 (Triangle inequality).

For any vectors

u, v

in an inner product space,

∥ u + v ∥ \leq ∥ u ∥ + ∥ v ∥

Proof.

We compute

∥ u + v ∥^{2} = ∥ u ∥^{2} + 2 ⟨ u, v ⟩ + ∥ v ∥^{2} \leq ∥ u ∥^{2} + 2∥ u ∥∥ v ∥ + ∥ v ∥^{2} = (∥ u ∥ + ∥ v ∥)^{2},

where the inequality follows from

⟨ u, v ⟩ \leq ∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥∥ v ∥

(the Cauchy–Schwarz inequality). Since both sides are nonnegative, taking square roots gives the result. □

3 Orthonormal Bases

Definition 6 (Orthonormal system and orthonormal basis).

A set

{e_{1}, \dots, e_{n}}

is called an orthonormal system if

⟨ e_{i}, e_{j} ⟩ = δ_{ij} = {10 if i = j, if i \neq = j .

An orthonormal system that is also a basis is called an orthonormal basis (ONB).

Theorem 7.

Every orthonormal system is linearly independent.

Proof.

Suppose

\sum_{i} c_{i} e_{i} = 0

. Taking the inner product with

e_{k}

gives

c_{k} = ⟨ \sum_{i} c_{i} e_{i}, e_{k} ⟩ = ⟨ 0, e_{k} ⟩ = 0

. □

Theorem 8 (Fourier expansion).

{e_{1}, \dots, e_{n}}

is an orthonormal basis for

V

, then every

v \in V

admits the expansion

v = i = 1 \sum n ⟨ v, e_{i} ⟩ e_{i} .

In other words, the

i

-th coordinate of

v

with respect to the orthonormal basis is simply

⟨ v, e_{i} ⟩

Proof.

Since

{e_{1}, \dots, e_{n}}

is a basis, we may write

v = \sum_{i = 1}^{n} a_{i} e_{i}

uniquely. Taking the inner product of both sides with

e_{k}

gives

⟨ v, e_{k} ⟩ = \sum_{i = 1}^{n} a_{i} ⟨ e_{i}, e_{k} ⟩ = \sum_{i = 1}^{n} a_{i} δ_{ik} = a_{k}

. □

4 The Gram–Schmidt Process

Theorem 9 (Gram–Schmidt orthonormalization).

Let

{v_{1}, \dots, v_{n}}

be a linearly independent set in an inner product space

V

. The following procedure produces an orthonormal set

{e_{1}, \dots, e_{n}}

Set $u_{1} = v_{1}$ and $e_{1} = u_{1} /∥ u_{1} ∥$ .
For $k = 2, \dots, n$ : set $u_{k} = v_{k} - \sum_{j = 1}^{k - 1} ⟨ v_{k}, e_{j} ⟩ e_{j}$ and $e_{k} = u_{k} /∥ u_{k} ∥$ .

Moreover,

span {e_{1}, \dots, e_{k}} = span {v_{1}, \dots, v_{k}}

at each stage.

Proof.

We proceed by induction on

k

. The case

k = 1

is clear. Suppose

{e_{1}, \dots, e_{k - 1}}

is an orthonormal basis for

span {v_{1}, \dots, v_{k - 1}}

. Define

u_{k} = v_{k} - \sum_{j = 1}^{k - 1} ⟨ v_{k}, e_{j} ⟩ e_{j}

. For each

j \leq k - 1

, we have

⟨ u_{k}, e_{j} ⟩ = ⟨ v_{k}, e_{j} ⟩ - ⟨ v_{k}, e_{j} ⟩ = 0

. Since

v_{k}

is not in

span {v_{1}, \dots, v_{k - 1}}

(by linear independence), we have

u_{k} \neq = 0

. Setting

e_{k} = u_{k} /∥ u_{k} ∥

completes the inductive step, and clearly

span {e_{1}, \dots, e_{k}} = span {v_{1}, \dots, v_{k}}

. □

5 Orthogonal Complements

Definition 10 (Orthogonal complement).

Let

W

be a subspace of an inner product space

V

. The orthogonal complement of

W

W^{⊥} = {v \in V ∣ ⟨ v, w ⟩ = 0 for all w \in W} .

Theorem 11.

Let

V

be a finite-dimensional inner product space and

W

a subspace of

V

. Then:

$W^{⊥}$ is a subspace of $V$ .
$V = W \oplus W^{⊥}$ .
$dim W^{⊥} = dim V - dim W$ .
$(W^{⊥})^{⊥} = W$ .

Proof.

(1) For

u_{1}, u_{2} \in W^{⊥}

and scalars

a, b

, we have

⟨ a u_{1} + b u_{2}, w ⟩ = a ⟨ u_{1}, w ⟩ + b ⟨ u_{2}, w ⟩ = 0

for all

w \in W

, so

a u_{1} + b u_{2} \in W^{⊥}

(2) Apply the Gram–Schmidt process to obtain an orthonormal basis

{e_{1}, \dots, e_{k}}

for

W

. Given any

v \in V

, set

w = \sum_{i = 1}^{k} ⟨ v, e_{i} ⟩ e_{i} \in W

. Then for each

j \leq k

⟨ v - w, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - ⟨ v, e_{j} ⟩ = 0

, so

v - w \in W^{⊥}

. Thus

v = w + (v - w)

gives

V = W + W^{⊥}

. The intersection

W \cap W^{⊥} = {0}

is immediate: if

u \in W \cap W^{⊥}

, then

⟨ u, u ⟩ = 0

, hence

u = 0

(3) From the direct sum decomposition,

dim V = dim W + dim W^{⊥}

(4) The inclusion

W \subseteq (W^{⊥})^{⊥}

follows from the definition. By (3),

dim (W^{⊥})^{⊥} = dim V - dim W^{⊥} = dim W

, so

W = (W^{⊥})^{⊥}

. □

6 Orthogonal Projection

Definition 12 (Orthogonal projection).

Given the decomposition

V = W \oplus W^{⊥}

, every

v \in V

can be written uniquely as

v = w + w^{⊥}

with

w \in W

and

w^{⊥} \in W^{⊥}

. The vector

w

is called the orthogonal projection of

v

onto

W

, denoted

proj_{W} (v) = w

Theorem 13.

{e_{1}, \dots, e_{k}}

is an orthonormal basis for

W

, then

proj_{W} (v) = i = 1 \sum k ⟨ v, e_{i} ⟩ e_{i} .

Proof.

Set

w = \sum_{i = 1}^{k} ⟨ v, e_{i} ⟩ e_{i} \in W

. For each

j = 1, \dots, k

⟨ v - w, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - i = 1 \sum k ⟨ v, e_{i} ⟩ ⟨ e_{i}, e_{j} ⟩ = ⟨ v, e_{j} ⟩ - ⟨ v, e_{j} ⟩ = 0,

v - w \in W^{⊥}

. The decomposition

v = w + (v - w)

is the direct-sum decomposition

V = W \oplus W^{⊥}

, whence

proj_{W} (v) = w

. □

Theorem 14 (Best approximation theorem).

The projection

proj_{W} (v)

is the closest point in

W

v

∥ v - proj_{W} (v) ∥ \leq ∥ v - w ∥ for all w \in W,

with equality if and only if

w = proj_{W} (v)

Proof.

Let

\overset{v}{^} = proj_{W} (v)

. For any

w \in W

∥ v - w ∥^{2} = ∥ v - \overset{v}{^} + \overset{v}{^} - w ∥^{2} = ∥ v - \overset{v}{^} ∥^{2} + ∥ \overset{v}{^} - w ∥^{2},

where the cross term vanishes because

v - \overset{v}{^} \in W^{⊥}

and

\overset{v}{^} - w \in W

are orthogonal. Since

∥ \overset{v}{^} - w ∥^{2} \geq 0

, we obtain

∥ v - w ∥^{2} \geq ∥ v - \overset{v}{^} ∥^{2}

, with equality only when

\overset{v}{^} = w

. □

Linear Algebra Algebra Textbook Inner Product Space Gram-Schmidt Orthogonal Complement

Folio Official

Mathematics "between the lines" — exploring the intuition textbooks leave out, written in LaTeX on Folio.

1 フォロワー·105 記事

Inner Product Spaces and Orthonormal Bases

1 The Definition of an Inner Product

2 The Cauchy–Schwarz Inequality

3 Orthonormal Bases

4 The Gram–Schmidt Process

5 Orthogonal Complements

6 Orthogonal Projection

あなたの専門知識を世界に発信しよう

Folio Officialの他の記事

Monetizing Your Articles — How to Earn Revenue Safely with Stripe Connect

Mastering Theorem Environments — theorem, definition, proof, and Friends

Combinatorial Designs and Latin Squares: Balanced Arrangements

Burnside's Lemma and P\'olya Enumeration: Counting Under Symmetry