Nicolas Flammarion

Achieving hallucination-free language generation with significantly reduced memory requirements reveals a sharp transition in capabilities that could redefine efficiency in language models.

Forgetting in language models can be optimized at the token level, leading to state-of-the-art performance in unlearning without the need for external supervision.

Learning rate decay, a common optimization technique, might be the culprit behind catastrophic forgetting in LLMs during fine-tuning.