Shuohuan Wang

Learned attention allocation patterns reveal that SWA is best positioned in lower layers, challenging conventional wisdom on attention distribution in LLMs.

Memento achieves state-of-the-art long-term subject consistency by reconstructing identities from memory, preventing dilution and loss across video shots.