Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
Addressing the evaluation overhead of discovering token dependencies in parallel masked generative models, this work analyzes whether sampling can achieve sublinear complexity over discrete distributions with latent forest dependencies. By sharing evaluations of hypothetical token reveals across dependence tests rather than fully recovering the underlying graph, the proposed sampler identifies safe parallel unmasking batches under bounded Hellinger oracle error. The algorithm guarantees total variation error within $\varepsilon$ with both total model queries and sequential depth scaling sublinearly as $O(N^C \varepsilon^a)$ for $C < 1$, while establishing polynomial lower bounds for irreversible product-commit baselines.
Masked generative models can break the linear evaluation barrier during parallel decoding without ever needing to fully reconstruct the underlying token dependency graph.
Masked generative models offer parallel token prediction, but accurate parallel sampling must account for dependencies among tokens. When dependencies are unknown, finding safe batches also costs model evaluations. We study whether total evaluations, including discovery, can be sublinear in sequence length $N$; sublinear sequential depth then follows. We consider discrete distributions with hidden forest structure, accessed through a fixed approximate conditional oracle. Under explicit regularity conditions and uniform Hellinger error bounds, for any fixed target accuracy $\varepsilon\in (0,1/8]$ and sufficiently large $N$, our sampler achieves seed-averaged total-variation error at most $\varepsilon$, with total masked-state submissions and sequential depth both bounded by $O(N^C \varepsilon^a)$ for constants $0 <C <1$ and $a > 0$. These guarantees use polynomial vocabulary size and an edge-response lower bound set by $N$ and $\varepsilon$. The sampler shares evaluations of hypothetical reveals across dependence tests to identify safe parallel batches without requiring full recovery of the hidden forest. A tunable parameter trades probing cost against irreversible commit rounds. In the same class, any admissible irreversible product-commit sampler attaining the same seed-averaged accuracy requires $Ω(N^c \varepsilon^b)$ counterfactual submissions or commit rounds in the worst case, for constants $c,b>0$.
RIKEN