Beren Millidge

Hybrid architectures can discard explicit position encodings like RoPE entirely because local mixing layers naturally inject a durable recency bias directly into global attention logits.

ZONOS2 8B sets a new standard in TTS with unmatched naturalness and voice fidelity, all while enhancing inference speed.

Larger models might delay plasticity loss, but they won't escape it—scaling alone isn't the answer for continual learning.