Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores a novel framework for collaborative mobile edge computing (MEC) aimed at optimizing large language model (LLM) inference under soft deadline constraints. By implementing a transformer-enhanced proximal policy optimization (PPO) approach, the authors effectively manage task dependencies and minimize deadline extensions, thereby improving the quality of service. Simulation results reveal that this method significantly increases task completion rates and overall system efficiency compared to traditional PPO and heuristic methods.
Transforming MEC for LLM inference, this approach boosts task completion rates while minimizing deadline extensions, even under strict latency constraints.
This paper investigates collaborative mobile edge computing (MEC) servers for large language model (LLM) inference under soft deadline constraints. In this system, to improve the quality of service, computations are expected to be completed within their deadlines. However, due to dependencies among tasks or subtasks, any missed deadline can lead to catastrophic consequences for the entire request. In this context, this work proposes an extended deadline mechanism with constrained flexibility. The main challenges lie in handling large-scale computations under strict latency constraints while limiting the number of allowable deadline extensions, especially in the presence of task dependencies within each request. To tackle these challenges, we develop a transformer-enhanced proximal policy optimization (PPO) framework that enables efficient collaboration among MEC servers. The proposed approach aims to maximize the number of tasks completed within their deadlines while minimizing the use of deadline extensions. By capturing temporal dependencies and cross-server interactions, the transformer improves decision-making for task migration. Simulation results demonstrate that the proposed method significantly outperforms conventional PPO and heuristic-based approaches in terms of task completion rate and overall system efficiency.