Search papers, labs, and topics across Lattice.
This paper geometrically re-derives Singular Value Decomposition (SVD) by analyzing how linear maps transform the unit circle into an ellipse, identifying orthogonal input directions that map to the ellipse's axes. This geometric construction is then shown to be the algorithmic foundation for Principal Component Analysis, kernel methods, and PageRank, with the proofs themselves translating directly into computational procedures. The approach reverses the typical spectral theorem-based justification, making SVD's connection to geometric transformations and its algorithmic implications explicit.
The geometric intuition behind SVD is revealed as the direct algorithmic engine driving PCA, kernel methods, and PageRank, unifying their proofs and implementations.
This article is a geometric rediscovery of the singular value decomposition, with a further claim: the construction it builds is the machinery behind much of machine learning. The same argument that answers an idle question about ellipses is the algorithm behind principal component analysis, kernel methods, and PageRank, and it is not only the results that transfer but the proofs themselves, run as procedures. The usual introduction states $A = U\Sigma V^T$ and justifies it via the spectral theorem applied to $A^T A$. This is correct but unilluminating, since it assumes a powerful theorem to reach a result that is, in the end, about ellipses. Part I reverses the order. A linear map sends the unit circle to an ellipse; one asks which input directions map to its axes, and finds, example after example, that they are perpendicular. In the plane this can be watched: rotate a frame, track how far its images are from perpendicular, and a sign change forces a frame where they are exactly perpendicular, which is also where the map stretches hardest. Maximizing the stretch and recursing generalizes this to n dimensions, with singular values falling out in order, and the construction proves the spectral theorem rather than assuming it. Part II puts each construction to work: maximize-and-recurse becomes the power method and PageRank; the lemma locating the maximizer becomes the stopping rule of gradient descent; the duality between $A^T A$ and $A A^T$ becomes the transport at the heart of kernel PCA. Each connection is stated with its boundary, saying what the decomposition supplies and where another idea takes over. Prerequisites are the standard sophomore sequence, and the worked examples are small enough to check by hand.
