Aleksandr Beznosikov

Prompt adaptation methods don't just change what LLMs say, they fundamentally rewire *how* they compute, with distinct mechanisms for text optimization, prompt tuning, and prefix tuning.

Two residual streams in diffusion language models halve the performance gains of adding a second stream by enabling distinct read and write routing patterns.

Interleaving expensive Muon spectral steps with cheap Lion sign descent not only cuts distributed training wall-clock time by 33%, but actually beats full Muon and AdamW on final perplexity.

LoRA-TSD achieves superior performance over all existing LoRA optimizers while offering the first global convergence guarantees for low-rank adaptations.

Riemannian optimization enables a new level of control over LLM refusal behavior, outperforming traditional methods that depend on auxiliary constructs.

Associative context retrieval can dramatically amplify the effectiveness of white-box attacks on LLMs without compromising their general performance.

Self-generated QA pairs can introduce significant biases, leading to fragile learning dynamics that may undermine model performance.

Aggressive 2-bit inference can backfire, leading to longer reasoning traces and accuracy drops, but targeted recovery methods can reverse this trend dramatically.

SGD's noise isn't Brownian motion, and this difference explains why neural nets diffuse along flat valleys in the loss landscape.

LoRA fine-tuning just got a memory-efficient upgrade: LoRSum matches or beats standard LoRA performance by reformulating optimization as a proximal problem and using diagonal K-FAC approximations, all without expensive SVD.