Projection on a line

Definition
Closed-form expression
Interpreting the scalar product

Definition

Consider the line in $mathbf{R}^n$ passing through $x_0 in mathbf{R}^n$ and with direction $u in mathbf{R}^n$ :

$left{ x_0+t u ~:~ t in mathbf{R} right} ,$

The projection of a given point on the line is a vector located on the line, that is closest to (in Euclidean norm). This corresponds to a simple optimization problem:

This particular problem is part of a general class of optimization problems known as least-squares. It is also a special case of a Euclidean projection on a general set.

Projection of the vector x=(1.6,2.28) on a line passing through the origin ( x_0=0 ) and with (normalized) direction u = (0.8944,0.4472) . At optimality the ‘‘residual’’ vector x-z is orthogonal to the line, hence z = tu , with t = x^Tu = 2.0035 . Any other point on the line is farther away from the point than its projection is.

The scalar t=u^Tx , ie the scalar product between and , is the component of along the normalized direction .

Closed-form expression

Assuming that is normalized, so that |u|_2 = 1 , the objecive function of the projection problem reads, after squaring:

$|x - x_0 - tu|_2^2 = t^2 - 2t u^T(x-x_0) + |x-x_0|_2^2 = (t - u^T(x-x_0))^2 + mbox{constant}.$

Thus, the optimal solution to the projection problem is

and the expression for the projected vector is

z^ast = x_0 + t^ast u = x_0 + u^T(x-x_0) u.

The scalar product u^T(x-x_0) is the component of x-x_0 along .

In the case when is not normalized, the expression is obtained by replacing with its scaled version u/|u|_2 :

$z^ast = x_0 + frac{u^T(x-x_0)}{u^Tu} u .$

Interpreting the scalar product

We can now interpret the scalar product between two non-zero vectors x,u , by applying the previous derivation to the projection of on the line of direction passing through the origin. If is normalized ( |u|_2=1 ), then the projection of on {bf L} is z^ast = (u^Tx)u . Its length is |z^ast|_2 = |u^Tx| . (See above figure.)

In general, the scalar product u^Tx is simply the component of along the normalized direction u/|u|_2 defined by .