### Linear Algebra for Graphics Geeks (SVD-VIII)

**The SVD and the Moore-Penrose Pseudoinverse**

Let's look at the SVD and its inverse.

**A = USV**

^{T}**A**

(V

VS

^{-1}= (USV^{T})^{- 1}=(V

^{T})^{-1}S^{-1}U^{-1}=VS

^{-1}U^{T}Here's the question for this post:

If we substitute the pseudoinverse of S for S

^{-1}in the product above, will we wind up with the pseudoinverse of A?

In other words, is

**A**?

^{+}= VS^{+}U^{T}The quick answer is yes.

I've read quite a bit on the subject in the past few months, but I don't recall seeing this worked out, so I'm going to do it just for the sake of sanity and entertainment. :)

A few reminders for the sake of clarity:

First, because U and V are orthogonal matrices their inverses and tranposes are the same, which is why either matrix cancels out its transpose (UU

^{T}=U

^{T}U=VV

^{T}=V

^{T}V=I). Second, there are several simplifications involving S and S

^{+}that exploit the identities of pseudoinverses. I'll note these when they are applied. Also, the inverse of a matrix product is the reverse product of the individual inverses (likewise with transposes). cf.

*The Matrix Cookbook*.

Matrix

**A**needs to meet four criteria to be a pseudoinverse. Let's substitute

**USV**for

^{T}**A**and

**VS**for

^{+}U^{T}**A+**and see if everything works out.

Rule #1:

**AA**

^{+}A = A(USV

^{T})(VS

^{+}U

^{T}) (USV

^{T}) =

US(V

^{T}V)S

^{+}(U

^{T}U) SV

^{T}=

U(SS

^{+}S)V

^{T}=

USV

^{T}

*note: The last step applies this rule (#1) to (SS*

^{+}S), which we can do because S+ is a pseudoinverse.Rule #2:

**A**

^{+}AA^{+}= A^{+}(VS

^{+}U

^{T})(USV

^{T})(VS

^{+}U

^{T}) =

VS

^{+}(U

^{T}U)S(V

^{T}V) S

^{+}U

^{T}=

V(S

^{+}SS

^{+})U

^{T}=

VS

^{+}U

^{T}

*note: The last step applies this rule (#2) to (S*

^{+}SS^{+}), which we can do because S+ is a pseudoinverse.Rule #3:

**(AA**

^{+})^{T}= AA^{+}AA

^{+}= (USV

^{T})(VS

^{+}U

^{T})

AA+ = USV

^{T}VS

^{+}U

^{T}

AA+ = US(V

^{T}V)S

^{+}U

^{T}

AA+ = USS

^{+}U

^{T}

AA+ = U(SS

^{+})U

^{T}

(AA+)

^{T}= (U(SS

^{+})U

^{T})

^{T}

(AA+)

^{T}= U

^{T}

^{T}(SS

^{+})

^{T}U

^{T}

(AA+)

^{T}= U(SS

^{+})

^{T}U

^{T}

(AA+)

^{T}= U(SS

^{+})U

^{T}

*note: The last step applies this rule (#3) to (SS*

^{+})^{T}, which we can do because S^{+}is a pseudoinverse.Rule #4:

**(A**

^{+}A)^{T}= A^{+}AA

^{+}A = (VS

^{+}U

^{T})(USV

^{T})

A

^{+}A = VS

^{+}(U

^{T}U)SV

^{T}

A

^{+}A = V(S

^{+}S)V

^{T}

(A

^{+}A)

^{T}= (V(S

^{+}S)V

^{T})

^{T}

(A

^{+}A)

^{T}= V

^{T}

^{T}(S

^{+}S)

^{T}V

^{T}

(A

^{+}A)

^{T}= V(S

^{+}S)

^{T}V

^{T}

(A

^{+}A)

^{T}= V(S

^{+}S)V

^{T}

*note: The last step applies this rule (#4) to (S*

^{+}S)^{T}, which we can do because S^{+}is a pseudoinverse.Finally, let's look at an identity mentioned in a my last post on the subject. This is an important one relating to least squares.

A

^{+}= (A

^{T}A)

^{-1}A

^{T}

A

^{+}= ((USV

^{T})

^{T}(USV

^{T}))

^{-1}(USV

^{T})

^{T}

A

^{+}= ((VS

^{T}U

^{T})(USV

^{T}))

^{-1}(VS

^{T}U

^{T})

A

^{+}= (USV

^{T})

^{-1}(VSU

^{T})

^{-1}(VSU

^{T})

A

^{+}= (USV

^{T})

^{-1}

A

^{+}= (V

^{T})

^{-1}S

^{+}U

^{-1}

A

^{+}= VS

^{+}U

^{T}

^{}

^{fin }

This is post #10 of Linear Algebra for Graphics Geeks. Here are links to posts #1, #2, #3, #4, #5, #6, #7, #8, #9.

Next: Some insights relating to rotation matrices.

## 0 Comments:

Post a Comment

Subscribe to Post Comments [Atom]

<< Home