Johannes von Oswald

Protein language models, like LLMs, suffer from a "Curse of Depth," where deeper layers contribute surprisingly little to the final prediction, suggesting opportunities for more efficient architectures.

Looping and depth-growing, two distinct methods for improving LLM reasoning, are actually two sides of the same iterative computation coin, and can be combined for even better results.