Robust Least-Squares

We start from the least-squares problem:

where $A in mathbf{R}^{m times n}$ , $y in mathbf{R}^m$ .

Now assume that the matrix is imperfectly known. A simple model is to assume that the matrix is only known to be within a certain ‘‘distance’’ (in matrix space) to a given ‘‘nominal’’ matrix hat{A} . Precisely, let us assume that assume |A-hat{A}| le rho , where |cdot| denotes the largest singular value norm, and rhoge0 measures the size of the uncertainty. We now address the robust least-squares problem:

$min_x : max_{|Delta A| le rho} : |(hat{A}+Delta A)x - y|_2.$

The interpretation of this problem is that it is trying to minimize the worst-case value of the residual norm.

For fixed , and using the fact that the Euclidean norm is convex, we have

$|(hat{A}+Delta A)x - y|_2 le |hat{A}x-y|_2 + |(Delta A) x|_2 .$

By definition of the largest singular value norm, and given our bound on the size of the uncertainty, we have

|(Delta A) x|_2 le |Delta A| |x|_2 le rho |x|_2.

Thus, we have a bound on the objective value of the robust problem:

$max_{|Delta A| le rho} : |(hat{A}+Delta A)x - y|_2 le |Ax-y|_2 + rho |x|_2.$

It turns out that the upper bound is attained by some choice of the matrix Delta A , specifically:

$Delta A = frac{rho}{|hat{A}x-y||_2 cdot |x|_2} (hat{A}x-y)x^T.$

Hence the robust least-squares is equivalent to the problem

The above is an SOCP:

$min_{x,u,v} : u + rho v ~:~ u ge |Ax-y|_2 , ;; v ge |x|_2.$

As given, this SOCP can be solved using SVD methods. However, problems involving constraints (such as sign constraints on ) cannot be solved by SVD, while they still can be solved with SOCP.