Dian-Hai Yu

Distributed attention can achieve near-zero communication overhead for mask localization and compute-communication overlap, unlocking massive throughput gains for long-context LLMs.

A novel attention mechanism slashes KV cache by 4x and training FLOPs by 7.9% while *improving* LLM quality, breaking the long-context "impossible triangle."

VESTA's policy-steered retrieval approach enables long-video agents to adaptively acquire and verify evidence, leading to substantial accuracy improvements over traditional methods.

Learned attention allocation patterns reveal that SWA is best positioned in lower layers, challenging conventional wisdom on attention distribution in LLMs.

PP-OCRv6 outperforms billion-scale VLMs on OCR tasks with a fraction of the parameters, achieving state-of-the-art accuracy and speed.