Search papers, labs, and topics across Lattice.
This paper introduces Maturing Markov Decision Processes (MMDPs), a novel framework that captures the asymmetry between increasing information and shrinking action sets in sequential decision-making. By employing an expiring-action priority principle, the authors demonstrate how to prioritize actions that must be resolved before advancing to the next decision stage. Experimental results reveal that this approach significantly enhances learning efficiency, particularly in complex decision environments, highlighting its scalability and practical relevance.
Explicitly modeling the asymmetry between information growth and action expiration can drastically improve decision-making efficiency in complex environments.
Sequential decision problems often exhibit an asymmetric evolution of information and decision flexibility: as a decision cycle unfolds, the agent receives richer information while feasible actions expire due to operational cutoffs, commitments, or resource constraints. Standard MDP formulations typically flatten this structure into stage-dependent state descriptions and action masks, thereby obscuring the nested information--action asymmetry that determines which decisions are urgent and which can be deferred. We introduce Maturing Markov Decision Processes (MMDPs), a formulation built around this information--action asymmetry. We characterize one of its key consequences through an expiring-action priority principle, which identifies the actions that must be resolved before the next stage. Motivated by this structure, we develop a structure-aware reinforcement learning framework with stage-aware policy design, expiring-action abstraction, and search-augmented learning with distillation. Experiments on a controlled multi-supplier replenishment problem, simplified cash-management environments of increasing complexity, and a production-scale simulator show that explicitly modeling this asymmetry improves learning efficiency and becomes increasingly valuable as decision problems scale.
