Mohammad R. Abu Ayyash

This simple architectural tweak—partitioning the residual stream and injecting a fixed profile—yields faster convergence and better perplexity than standard Transformers.

LLMs can learn transferable cognitive skills, not just domain-specific facts, as demonstrated by a routing mechanism that combines specialized adapter stacks to solve unseen tasks.