Search papers, labs, and topics across Lattice.
This paper addresses the challenge of ensuring strict end-to-end peak latency guarantees in delay-sensitive network applications by introducing a novel network control framework that combines Effective Congestion (EC) metrics and Multi-Agent Deep Reinforcement Learning Effective Congestion (MADRL EC ($p^*$)). The proposed approach utilizes a unified training objective that integrates various policy-learning paradigms, leading to the development of the Model-Guided Annealed Reinforcement Learning (MGA-RL) protocol, which enhances sample efficiency and reduces training time. Key results demonstrate that this hybrid architecture significantly improves load-balancing and traffic management, making it deployable in real-world scenarios with stringent latency requirements.
A novel deadline-aware metric and a hybrid reinforcement learning architecture dramatically improve network control efficiency, enabling timely delivery of critical information in dynamic environments.
Timely delivery of delay-sensitive information over dynamic, heterogeneous networks is essential for NextG interactive applications, yet providing strict End-to-End (E2E) peak latency guarantees remains an open challenge. Two obstacles limit the adoption of learning-based network control in this setting: traditional volume-based routing metrics, while highly effective for general traffic management, are not designed to capture traffic urgency; and Deep Reinforcement Learning (DRL) controllers trained from scratch suffer from sample inefficiency, long training times, and early-stage exploration volatility. This paper introduces a deployment-focused network control framework that addresses both obstacles. First, we present Effective Congestion (EC), a deadline-aware metric family that quantifies interface congestion by packet urgency and proactively filters non-viable traffic, coupled with a Uniform Path Grouping (UPG) distribution heuristic promoting robust load-balancing; the resulting policies are embedded into Multi-Agent Deep Reinforcement Learning Effective Congestion ($p^*$) (MADRL EC ($p^*$)), a hybrid architecture combining a distributed scheduler with a centralized RL-based router. Second, we introduce a unified training objective that generalizes existing policy-learning paradigms---behavioral cloning, offline Reinforcement Learning (RL), online RL, and offline-to-online schemes---as special cases, combining a live-reward term, a pre-collected-reward term, and a policy-imitation term. From this objective, we derive the Model-Guided Annealed Reinforcement Learning (MGA-RL) protocol, instantiated on a Deep Deterministic Policy Gradient (DDPG) backbone: a deployment-oriented, demonstration-driven training approach that generalizes conventional Offline-to-Online (O2O) schemes, in which trajectories from a lightweight [...]
