Search papers, labs, and topics across Lattice.
This paper introduces a method for memory-efficient inference called Memory-Sovereign Inference, which separates various resource demands and certifies exact execution in complex environments. By analyzing a specific Qwen3-Next identity, the authors demonstrate that their approach can exceed traditional memory limits while maintaining exactness in output, achieving a lower bound of 43.59375 GiB for semantic demand. The results show that their method can efficiently manage resources without compromising on execution accuracy, significantly enhancing the understanding of memory management in AI models.
Achieving a semantic demand lower bound of 43.59375 GiB reveals that traditional memory limits can be surpassed without sacrificing execution accuracy in AI inference.
Storage-backed inference is easy to overclaim: process RSS excludes charged page cache, process-local device readings exclude board-wide use, and successful generation does not establish correct asynchronous reuse. We present a falsifiable certificate separating representation, semantic demand, scheduler requests, and traffic while naming resource authorities, exactness horizons, and tested reuse transitions. At one Qwen3-Next identity, the stock router selects all 48 x 512 managed layer-expert objects during 32K prefill. Their duplicate-free, overlap-free canonical union gives a 43.59375 GiB semantic-demand lower bound, exceeding the declared 34 GiB full-residency envelope and the 11 GiB host-hard plus physical 24 GiB-device envelope. LRU64 execution stays within its host-hard/GPU-audited contract. Against one prespecified zero-cache oracle, all 64 token IDs, every byte of 64 complete 151,936-float logit rows, 3,408 route events, response bytes, and recorded consumer and destination identities are exact. Recurrent-state and upstream-runtime equality are excluded. In a matched source campaign, the buffered path completes exactly but reaches the 11 GiB host ceiling and records 33,481 memory.max events. The blocking one-window direct path and complete eight-window asynchronous component are exact with positive margin and zero limit events. Across six counterbalanced pairs, the complete asynchronous component takes 32.3% of the blocking direct path's wall time at identical physical source bytes per output; the comparison jointly changes queue depth, overlap, and lifecycle implementation. Separately, fourteen prespecified control/fault cells pass across later Qwen3-Next and Gemma 4 binaries, supporting fail-closed behavior only for the named transitions. The principal experiment is one fixed model, workload, runtime, device, and 64-output horizon.
