Linear Matrix Inequalities

SDP > Conic problems | LMIs | Standard Forms | Applications

Definition
LMIs and convex sets
Special cases

Definition

Positive Semi-Definite Matrices

Recall from here that a n times n symmetric matrix is positive semi-definite (PSD) if and only if every one of its eigenvalues is non-negative. We use the notation F succeq 0 to mean that is PSD.

An alternative condition for to be PSD is that the associated quadratic form is non-negative:

$forall : z in mathbf{R}^n ~:~ z^T F z ge 0.$

The set of PSD matrices is convex, since the conditions above represent (an infinite number of) ordinary linear inequalities on the elements of the matrix .

Examples:

A simple example.
For any vector $v in mathbf{R}^n$ , the dyad is PSD, since the associated quadratic form is .
More generally, for any rectangular matrix , the ‘‘square’’ matrix is PSD.
The converse is true: any PSD matrix can be factored as for some appropriate matrix .

Standard form

A linear matrix inequality is a constraint of the form

$F_0 + sum_{i=1}^m x_i F_i succeq 0,$

where the n times n matrices F_0,ldots,F_m are symmetric.

An LMI in two variables: F(x) := x_1 F_1 + x_2 F_2 preceq I , where F_1,F_2 are the two 5 times 5 symmetric matrices

$tiny begin{array}{c} F_1 = left(begin{array}{ccccc}-1.3 & -4.2 & -0.1 & 2.1 & -1 -4.2 & -0.1 & -1.7 & -4.5 & 0.9 -0.1 & -1.7 & 2.3 & -4.4 & -0.4 2.1 & -4.5 & -4.4 & 3.3 & -1.7 -1. & 0 0.9 & -0.4 & -1.7 & 4.7 end{array}right), F_2 = left(begin{array}{ccccc} 1.6 & 3.9 & 1.6 & -5.3 & -4. 3.9 & -1.8 & -4.7 & 1. & 0 2.9 1.6 & -4.7 & -1.3 & 1.6 & -2.6 -5.3 & 1. & 0 1.6 & 2.7 & 2.6 -4. & 0 2.9 & -2.6 & 2.6 & -3.4 end{array}right). end{array}$

The matrices F_0,ldots,F_m are referred to as the coefficient matrices. Sometimes, these matrices are not explicitly defined. That is, if $F : mathbf{R}^m rightarrow mathbf{S}^n$ is an affine map that takes its values in the set of symmetric matrices of order , then F(x) succeq 0 is an LMI.

An alternate form for LMIs is as the intersection of the positive semi-definite cone with an affine set:

where {cal A} is affine. The form we have seen before, and the above one, are equivalent, in the sense that we can always transform one into the other (at the expense possibly of adding new variables and constraints).

LMIs and Convex Sets

Let us denote by mathbf{X} the set of points $x in mathbf{R}^m$ that satisfy the above LMI.

$mathbf{X} := left{ x ~:~ F_0 + sum_{i=1}^m x_i F_i succeq 0 right}.$

The set mathbf{X} is convex. Indeed, we have F(x) succeq 0 if and only if

$forall : z in mathbf{R}^n ~:~ z^T F(x) z ge 0.$

Since

$z^TF(x)z = f_0(z) + sum_{i=1}^m x_i f_i(z)$

with f_i(z) := z^TF_iz , i=0,ldots,m , we obtain that the condition on : F(x) succeq 0 can be represented as an intersection of (an infinite number of) half-space conditions.

Supporting hyperplanes: The picture shows how the set seen above can be interpreted as the intersection of a number of half-spaces. Each half-space corresponds to an affine inequality of the form z^T(I-F(x))z^T ge 0 , for a specific vector . The vectors are chosen so that the line touches the boundary of the LMI set, so that it defines a supporting hyperplane.

Multiple LMIs

We can combine multiple LMIs into one. Consider two affine maps from mathbf{R} to a space of symmetric matrices of order n_1,n_2 , $F_1: mathbf{R}^m rightarrow mathbf{S}^{n_1}$ , $F_2: mathbf{R}^m rightarrow mathbf{S}^{n_2}$ . Then the two LMIs

are equivalent to one LMI, involving a larger matrix of size (n_1+n_2) times (n_1+n_2) :

$left(begin{array}{cc} F_1(x) & 0 0 & F_2(x) end{array}right) succeq 0 .$

This corresponds to intersecting the two LMI sets.

Special Cases

LMIs include as special cases the following.

Ordinary affine inequalities

Consider single affine inequality in $x in mathbf{R}^n$ :

where $a in mathbf{R}^n$ , b in mathbf{R} . (The above set describes a half-space.) The above is a trivial special case of LMI, where the coefficient matrices are scalar: F_0 = b , F_i = -a_i , i=1,ldots,n .

Using the result above on multiple LMIs, we obtain that the set of ordinary affine inequalities

can be cast as a single LMI F(x) succeq 0 , where

$F(x) = left(begin{array}{ccc} b_1-a_1^Tx & & & ddots & && b_m -a_m^Tx end{array}right) .$

Second-order cone inequalities

Second-order cone (SOC) inequalities can be represented as LMIs. To see this, let us start with the ‘‘basic’’ SOC |y|_2 le t , with $y in mathbf{R}^n$ and t in mathbf{R} . The SOC can be represented as

$left(begin{array}{cc} t & y^T y & t I_n end{array}right) = left(begin{array}{cccc} t & y_1 & ldots & y_n y_1 & t & & vdots & &ddots& y_n & & & t end{array}right) succeq 0.$

Indeed, we check that for every $z in mathbf{R}^n$ , alpha in mathbf{R} , we have

$left(begin{array}{c} alpha z end{array}right)^T left(begin{array}{cc} t & y^T y & tI_n end{array}right) left(begin{array}{c} alpha z end{array}right) = |t z+ alpha y|_2^2 + alpha^2 (t-|y|_2^2) ge 0$

if and only if |y|_2 le t .

More generally, a second-order cone inequality of the form

with $Ainmathbf{R}^{m times n}$ , $b in mathbf{R}^m$ , $c in mathbf{R}^n$ , $d in mathbf{R}^n$ , can be written as the LMI

$left(begin{array}{cc} c^Tx+d & (Ax+b)^T (Ax+b) & (c^Tx+d) I_n end{array}right) succeq 0.$

The proof relies on the Schur complement lemma.