Search papers, labs, and topics across Lattice.
This paper introduces a unified framework for defining and analyzing behavioral structures in reinforcement learning, focusing on state abstraction in complex systems. By establishing a compositional method to specify behavioral semantics through local, one-step descriptions, the authors demonstrate how behavioral structures can be safely transferred between abstract and concrete systems. Key results include the development of quantitative metrics derived from logical behavioral semantics, providing a robust foundation for reasoning about behaviors under state abstraction.
A unified framework reveals how to safely transfer behavioral structures in reinforcement learning, bridging the gap between abstract and concrete systems.
State abstraction plays a key role in scaling reinforcement learning to complex but structured systems. In studying such systems, a wide range of behavioral structures have been studied in reinforcement learning, including value functions, invariants, bisimulation relations, and behavioral metrics. However, a general principle for determining what structures are provably preserved under state abstraction is still lacking. In this paper, we present a unified framework for defining and analyzing behavioral structures in reinforcement learning. Our framework provides a compositional way to specify behavioral semantics based on local, one-step descriptions of system dynamics. Using this framework, we establish results showing how behavioral structures can be safely transferred between abstract and concrete systems. We further show how to construct quantitative metrics from logical behavioral semantics with soundness guarantees. Together, these results provide a principled foundation for reasoning about behaviors under state abstraction in reinforcement learning and offer reusable definition and proof principles for a broad class of behavioral structures in reinforcement learning.
