Boxing Chen

MATCH reveals that integrating in-context retrieval can dramatically boost the performance of sparse attention models without sacrificing efficiency.

Static rankings of attention heads for local/global behavior become unreliable after hybridizing attention mechanisms in LLMs, necessitating adaptive selection methods like BOSCH.