Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the invariance of representation-derived measurements under function-preserving reparameterizations in language models, revealing that column-permutation parallel analysis fails to maintain this invariance due to changes in reference distribution and component counts. Empirical results across five models and three retrieval domains demonstrate significant discrepancies in component counts and decision-making, highlighting that these measurements can be influenced by hidden-coordinate choices rather than the model's intrinsic properties. The findings underscore the importance of using orthogonally invariant comparator scores, which maintain numerical stability and consistent discrimination across transformations.
Column-permutation parallel analysis can mislead researchers by reflecting hidden-coordinate choices instead of true model properties, with median component count disagreements reaching 0.79.
Hidden coordinates are not uniquely determined by a language model's input--output function, so representation-derived measurements should be invariant to function-preserving changes of basis. This study shows that column-permutation parallel analysis violates function-preserving reparameterization invariance because its reference distribution and selected component count can change while the model function and observed covariance spectrum remain fixed. More generally, a data-internal reference procedure cannot simultaneously preserve every coordinate marginal, remain orthogonally equivariant, and remove cross-coordinate covariance. Empirically, across five models, three retrieval domains, and 75 transformations, median component-count disagreement is 0.79 and median fixed-threshold decision disagreement is 0.26. A centering-only control isolates the reference-driven effect, with 1,141 of 1,200 component counts changing despite an unchanged observed spectrum, whereas independent parallel analysis seeds change none of the corresponding decisions. By contrast, orthogonally invariant comparator scores remain numerically stable with similar held-out discrimination. Together, these results show that parallel analysis-derived component counts and decisions can reflect hidden-coordinate choice rather than a well-defined property of the model.