In mathematics, and in particular linear algebra, the pseudoinverse $A^+$ of an $m\; \backslash times\; n$ matrix is a generalization of the inverse matrix . More precisely, this article talks about the Moore-Penrose pseudoinverse, which was apparently independently described by E. H. Moore and Roger Penrose. A common use of the pseudoinverse is as an approximate or 'best' (least squares) solution to a system of linear equations (see below under Applications). The pseudoinverse is defined and unique for all matrices whose entries are real or complex numbers. Usually, the pseudoinverse is computed using singular value decomposition.

Definition

---

$A^+$ is the unique matrix which satisfies the following criteria:

1. $A\; A^+A\; =\; A;$
2. $A^+A\; A^+\; =\; A^+$       ($A^+$ is a weak inverse for the multiplicative semigroup.)
3. $(AA^+)^*\; =\; AA^+$       (That is, $AA^+$ is Hermitian.)
4. $(A^+A)^*\; =\; A^+A$       ($A^+A$ is also Hermitian.)

Here $M^*$ is the conjugate transpose of a matrix M.

An alternative way to define the pseudoinverse is via a limiting process:

$A^+\; =\; \backslash lim\_\{\backslash delta\; \backslash to\; 0\}\; (A^*\; A\; +\; \backslash delta\; I)^\{-1\}\; A^*$

= \lim_{\delta \to 0} A^* (A A^* + \delta I)^{-1} (see Tikhonov regularization). These limits exist, even if $(A\; A^*)^\{-1\}$ and $(A^*\; A)^\{-1\}$ do not exist.

Properties

---

• Pseudoinversion is reversible, it is its own inverse:
  $(A^+)^+\; =\; A$
• The pseudoinverse of a zero matrix is its transpose:
  $0^+\; =\; 0^T\; .$
• Pseudoinversion commutes with transposition, conjugation and taking the conjugate transpose:
  $(A^T)^+\; =\; (A^+)^T$,
  $\backslash overline\{A\}^+\; =\; \backslash overline\{A^+\}$ and
  $(A^*)^+\; =\; (A^+)^*\; .$
• The pseudoinverse of a scalar multiple of $A$ is the reciprocal multiple of $A^+$:
  $(\backslash alpha\; A)^+\; =\; \backslash alpha^\{-1\}\; A^+$, for $\backslash alpha$ ≠ 0 .
• If the pseudoinverse of $A^*A$ is already known, it may be used to compute $A^+$:
  $A^+\; =\; (A^*A)^+A^*$ .
• Likewise if $(AA^*)^+$ is already known:
  $A^+\; =\; A^*(AA^*)^+$ .

Special cases

---

If the columns of $A$ are linearly independent, then $A^*\; A$ is invertible. In this case, an explicit formula is ^*

$A^+\; =\; (A^*\; A)^\{-1\}\; A^*.$

In the first limit expression above, the $\backslash delta$ summand vanishes. It follows that $A^+$ is a left inverse of A:   $A^+\; A\; =\; I$ .

If the rows of $A$ are linearly independent, then $A\; A^*$ is invertible. In this case, an explicit formula is

$A^+\; =\; A^*(A\; A^*)^\{-1\}.$

In the second limit expression above, the $\backslash delta$ summand vanishes. It follows that $A^+$ is a right inverse of A:   $A\; A^+\; =\; I$ .

If both columns and rows are linearly independent (that is, for square nonsingular matrices), the pseudoinverse is just the inverse:

$A^+\; =\; A^\{-1\}\; .$

If A and B are such that the product $AB$ is defined and either A or B is unitary, then $(AB)^+\; =\; B^+\; A^+$.

It is also possible to define a pseudoinverse for scalars and vectors. This amounts to treating these as matrices. The pseudoinverse of a scalar $x$ is zero if $x$ is zero and the reciprocal of $x$ otherwise:

\\ x^{-1}, & \mbox{otherwise}. \end{matrix}\right.

The pseudoinverse of the null vector is the transposed null vector. The pseudoinverse of other vectors is the conjugate transposed vector divided by its squared magnitude:

\\ {x^* \over x^* x}, & \mbox{otherwise}. \end{matrix}\right.

For proof, simply check that these definitions meet the defining criteria for the pseudoinverse.

Derivation

---

The pseudoinverse is used to find the best solution (in a least squares sense) to systems of linear equations which are either overdetermined or underdetermined.

If $(A^*\; A)^\{-1\}$ exists,

$Ax\; =\; b$

$A^*\; A\; x\; =\; A^*\; b$

$(A^*\; A)^\{-1\}(A^*\; A)\; x\; =\; (A^*\; A)^\{-1\}A^*\; b$

$x\; =\; (A^*\; A)^\{-1\}A^*\; b$

which yields the pseudoinverse

$A^+\; =\; (A^*\; A)^\{-1\}A^*$ .

Finding the pseudoinverse of a matrix

---

Let k be the rank of a $m\; \backslash times\; n$ matrix A. Then A can be decomposed as $A\; =\; BC$, where B is a $m\; \backslash times\; k$-matrix and C is a $k\; \backslash times\; n$ matrix. Then

$$

A^+ = C^*(CC^*)^{-1}(B^*B)^{-1}B^*.

If A has full row rank, so that k = m, then B can be chosen to be the identity matrix and the formula reduces to $A^+\; =\; A^*(AA^*)^\{-1\}$. Similarly, if A has full column rank (that is, k = n), we have $A^+\; =\; (A^*A)^\{-1\}A^*.$

A computationally simpler way to get the pseudoinverse is using the singular value decomposition (SVD).

If $A\; =\; U\backslash Sigma\; V^*$ is the singular value decomposition of A, then $A^+\; =\; V\backslash Sigma^+\; U^*.$ For a diagonal matrix such as $\backslash Sigma$, we get the pseudoinverse by taking the reciprocal of each non-zero element on the diagonal.

If a pseudoinverse is already known for a given matrix, and the pseudoinverse is desired for a related matrix, the pseudoinverse for the related matrix can be computed using specialized algorithms that may need less work. In particular, if the related matrix differs from the original one by only a changed, added or deleted row or column, incremental algorithms exist that exploit the relationship.

Applications

---

The pseudoinverse provides a least squares solution to a system of linear equations (SLE) .

Given a SLE $A\; x\; =\; b$, we look for a vector $x$ that minimizes $\backslash |A\; x\; -\; b\backslash |^2$, where $\backslash |\backslash ,\backslash cdot\backslash ,\backslash |$ denotes the Euclidean norm.

The general solution to an inhomogeneous SLE $A\; x\; =\; b$ is the sum of a particular solution of the inhomogeneous system and the general solution of the corresponding homogeneous system $A\; x\; =\; 0$.

Lemma: If $(A\; A^*)^\{-1\}$ exists, then the solution $x$ can always be written as the sum of the pseudoinverse solution of the inhomogeneous system and a solution of the homogeneous system:

$x\; =\; A^*(A\; A^*)^\{-1\}b\; +\; (1\; -\; A^*(A\; A^*)^\{-1\}\; A)y.$

Proof:

{| $Ax$ $=$ $A\; A^*(A\; A^*)^\{-1\}$ $b$ $+$ $A\; y\; -\; A\; A^*(A\; A^*)^\{-1\}\; A\; y$ $=$ $b$ $+$ $A\; y\; -\; A\; y$ $=$ $b$ .

Here, the vector $y$ is arbitrary (apart from the dimensionality). In both summands, the pseudoinverse $A^*(A\; A^*)^\{-1\}$ appears. If we write it as $A^+$, the equation looks like this:

$x\; =\; A^+\; b\; +\; (1\; -\; A^+\; A)y.$

The first summand is the pseudoinverse solution. In the sense of the least squares error, it is the best linear approximation to the actual solution. This means that the correction summand has minimal euclidean norm. The second summand represents a solution of the homogeneous system $A\; x\; =\; 0$, because $(1\; -\; A^+\; A)$ is the projection on the kernel (null space) of A, while $(A^+A)\; =\; A^*\; (A\; A^*)^\{-1\}\; A$ is the projection onto the image (range) of A (the space spanned by the column vectors of A).

References

---

1. Adi Ben-Israel, Thomas N.E. Greville: Generalized Inverses. ISBN 0387002936, Springer-Verlag (2003)
2. E. H. Moore: On the reciprocal of the general algebraic matrix. Bulletin of the American Mathematical Society 26, 394-395 (1920)
3. Roger Penrose: A generalized inverse for matrices. Proceedings of the Cambridge Philosophical Society 51, 406-413 (1955)
4. Roger Penrose: On best approximate solution of linear matrix equations. Proceedings of the Cambridge Philosophical Society 52, 17-19 (1956)

External links

---

• Pseudoinverse on PlanetMath
• Pseudoinverse on MathWorld
• Moore-Penrose Inverse on MathWorld

Matrix theory | 擬似逆行列