Riccardo Grazzi

This work shows that Kimi Delta Attention (KDA) can realize 2D rotations by combining a single delta-rule transformation with a second reflection supplied by its channel-wise gate, and characterize the expressivity of CKDA and prove that every orthogonal diagonal-plus-rank-one matrix is exactly a CKDA transition matrix.

Linear RNNs beat Transformers at state-tracking when the task is framed as code execution, but surprisingly falter when actions are only partially observable.