Yuji Yamamoto

Contrary to widespread assumptions, RoPE is not responsible for attention sinks and outlier activations; the true culprits are causal-mask self-concentration and value-non-mixing at the sequence start.

Mamba's per-word processing time aligns with human reading speeds, offering a new perspective on cognitive language processing.

Silent divergence in LLM outputs can mask adversarial bit flips, allowing undetected exploitation of shared KV-cache vulnerabilities.