Vincent Gripon

Unlock hidden performance in your pre-trained language models with "inner looping," a simple inference-time trick that repeatedly refines latent representations by re-applying selected transformer blocks.

Transformers suffer from a subtle but significant misalignment: residual connections inadvertently tie information to the *wrong* token, but a simple residual attenuation fix can boost performance.

Skip the training: MUKA, a multi-kernel adaptation framework, lets Large Audio-Language Models achieve state-of-the-art few-shot performance without any additional training.