Length and Orthogonality in Rⁿ

The dot product measured angles between arrows in the plane. The same formula works in $\mathbb{R}^n$ and brings geometry (length, distance, right angles) into every dimension. This unit builds on it all the way to data fitting.

For $\mathbf{u}, \mathbf{v}$ in $\mathbb{R}^n$, the inner product (dot product) is $\mathbf{u}\cdot\mathbf{v}$ $= \mathbf{u}^{T}\mathbf{v}$ $= u_1v_1 + \cdots + u_nv_n$. The length (norm) is $\lVert\mathbf{v}\rVert = \sqrt{\mathbf{v}\cdot\mathbf{v}}$, the distance between $\mathbf{u}$ and $\mathbf{v}$ is $\lVert\mathbf{u} - \mathbf{v}\rVert$, and a unit vector has length $1$: dividing any nonzero $\mathbf{v}$ by its length gives the unit vector in its direction.

A set of vectors is orthogonal if every pair in it is orthogonal. Nonzero orthogonal vectors are automatically independent: if $c_1\mathbf{u}_1 + \cdots + c_p\mathbf{u}_p = \mathbf{0}$, dot both sides with $\mathbf{u}_1$; every term but the first vanishes, leaving $c_1\lVert\mathbf{u}_1\rVert^{2} = 0$, so $c_1 = 0$, and likewise for every $c_i$. That is why orthogonal bases are so convenient: each weight can be found by one dot product.

In higher dimensions the dot product is just as concrete. Two survey answers, two documents' word counts, or two days of stock returns are vectors in $\mathbb{R}^{n}$, and the angle between them (through $\cos\theta = \frac{\mathbf{u}\cdot\mathbf{v}}{\lVert\mathbf{u}\rVert\lVert\mathbf{v}\rVert}$) is a standard measure of how similar they are. Orthogonal means "no overlap at all".