Search papers, labs, and topics across Lattice.
This paper investigates Diffusion-MPC within the discrete Tetris environment, employing a MaskGIT-style denoiser to sample placement sequences and reranking to select actions. The study analyzes the impact of feasibility masking, reranking strategies (heuristic, DQN critic, hybrid), and compute scaling (candidate count and planning horizon) on performance. Key findings include the necessity of feasibility masking for improved performance and survival, the misalignment of naive DQN reranking with rollout quality, and the detrimental effects of longer planning horizons due to uncertainty compounding.
Diffusion-based planners in discrete environments like Tetris can actually *degrade* performance with longer planning horizons or naive DQN-based action selection, highlighting the need for careful consideration of uncertainty compounding and critic alignment.
We study diffusion-based model predictive control (Diffusion-MPC) in discrete combinatorial domains using Tetris as a case study. Our planner samples candidate placement sequences with a MaskGIT-style discrete denoiser and selects actions via reranking. We analyze three key factors: (1) feasibility-constrained sampling via logit masking over valid placements, (2) reranking strategies using a heuristic score, a pretrained DQN critic, and a hybrid combination, and (3) compute scaling in candidate count and planning horizon. We find that feasibility masking is necessary in discrete domains, removing invalid action mass (46%) and yielding a 6.8% improvement in score and 5.6% improvement in survival over unconstrained sampling. Naive DQN reranking is systematically misaligned with rollout quality, producing high decision regret (mean 17.6, p90 36.6). Shorter planning horizons outperform longer ones under sparse and delayed rewards, suggesting uncertainty compounding in long imagined rollouts. Overall, compute choices (K, H) determine dominant failure modes: small K limits candidate quality, while larger H amplifies misranking and model mismatch. Our findings highlight structural challenges of diffusion planners in discrete environments and provide practical diagnostics for critic integration.
