Claire Boyer

Softmax attention selectively exploits latent structure in Gaussian mixtures, solving nonlinear tasks beyond linear attention's reach while matching linear performance on tractable problems.

This work shows that a direction-dependent schedule decomposes into two independent design choices: a variance path, which fully determines the intermediate laws and probability flow, and a factorization, which leaves this flow unchanged while controlling irreducible regression variance.