Jason D. Lee

This work proposes Abstract Token Curriculum (ATC), a novel curriculum learning framework that elicits effective continuous intermediate representations without direct supervision or manual scratchpad design and shows ATC's effectiveness on graph reachability and arithmetic learning tasks.

Retaining the right contextual information can boost long-context modeling performance by over 20% compared to traditional methods.

Logarithmic-depth networks can efficiently learn complex Boolean functions that constant-depth networks cannot, revealing a critical algorithmic separation in neural network capabilities.