Jie Sun

Repurposing speculative decoding backbones for output-length prediction slashes short-request latency by nearly 35%.

IDA-OPD not only boosts diversity in distilled models but does so while cutting computational costs, challenging the notion that more data always leads to better performance.

EasyOPD unifies on-policy distillation methods, enabling seamless integration and superior performance across diverse tasks in large language models.

LLMs choke on long numerical sequences, but a simple separator token trick can boost accuracy by 35% and cut token costs by 16%—without any training.