Up to this point in a linear algebra course, the only operations available in a vector space are addition and scalar multiplication. There is no notion of "length," no notion of "angle," and no way to say whether two vectors are "perpendicular." All of that changes with a single additional structure: the inner product.

1 The definition

The standard inner product on ${\mathbb{R}}^n$, $⟨\mathbf{u},\mathbf{v}⟩=\sum u_i{v}_i$, is the most familiar — but it is far from the only one.

2 Length and angle

An inner product immediately yields:

When $⟨\mathbf{u},\mathbf{v}⟩=0$, we have $\theta =90°$, and the vectors are orthogonal.

This inequality is what makes the definition of $\cos \theta$ legitimate: it guarantees $|\cos \theta |\le 1$.

3 Gram–Schmidt orthonormalization

Given any basis, the Gram–Schmidt process produces an orthonormal one — a basis whose vectors are mutually perpendicular and each of unit length.

At each step, the idea is the same: strip away the components along directions already chosen. What remains is necessarily orthogonal to all of them.

4 Orthogonal projection: finding the closest point

One of the most powerful applications of inner products is orthogonal projection.

Given a subspace $W$ and a vector $\mathbf{v}$, the orthogonal projection $\hat{\mathbf{v}}$ onto $W$ is the point in $W$ closest to $\mathbf{v}$.

5 Least squares

Suppose you have experimental data $(x_1,y_1),\dots ,(x_m,y_m)$ and want to fit a line $y=a+bx$. When $m>2$, there is generally no line passing through all points — the system $A\mathbf{x}=\mathbf{b}$ is overdetermined.

The least-squares solution is the one that minimizes $\Vert A\mathbf{x}-\mathbf{b}{\Vert }^2$. Geometrically, it projects $\mathbf{b}$ onto the column space of $A$ and then solves the consistent system.

6 The bridge to Fourier analysis

Equip $C[-\pi ,\pi ]$ with the inner product $⟨f,g⟩={\int }_{-\pi }^{\pi }f(x)g(x)dx$. The functions $\{1,\cos x,\sin x,\cos 2x,\sin 2x,\dots \}$, suitably normalized, form an orthonormal system.

Projecting a function $f$ onto these basis functions gives the Fourier expansion:

Fourier series is orthogonal projection in an infinite-dimensional inner product space. The same principle that finds the best-fit line through three data points also decomposes a sound wave into its constituent frequencies.

7 The takeaway

An inner product gives a vector space geometry: length, angle, and orthogonality. With these come the Gram–Schmidt process, orthogonal projection, least squares, and Fourier analysis — all manifestations of a single idea. Whether you are fitting a regression line to data or decomposing a function into harmonics, you are projecting onto a subspace in an inner product space. One structure, one principle, and an extraordinary range of applications.