Marco Mondelli

Mixing auxiliary data doesn't just offer more tokens—under the right spectral alignment and dataset growth ratio, it provably accelerates the downstream power-law scaling exponent itself.

Self-supervised projection heads mask severe dimensional collapse in the upstream backbone, a hidden failure mode that can be eliminated by enforcing spectral weight balance directly across encoder layers.

Smaller initialization scales in early layers unlock superior fine-tuning generalization by enabling both feature reuse and refinement, challenging the intuition that larger initializations are always better.