G. Kaissis

It is shown that predictions on a patient's unseen future data can change significantly if a model observed that same patient's anonymised historical data during training, a phenomenon known as memorisation bias, which exists across diverse data modalities and model architectures, and over prolonged time spans.

Achieving up to 99% of the theoretical maximum speed-up in looped language models could revolutionize inference efficiency in AI applications.

Looping a language model block four times only gives you the effective capacity of 1.4 additional unique blocks, but costs as much to train as 2.4.