Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces EdgeCoInfer, a framework designed to optimize the deployment of Multimodal Large Language Models (MLLMs) in resource-constrained edge environments by enabling adaptive granularity in model sharing and partitioning. The authors tackle the complex Mixed-Integer Non-Linear Programming (MINLP) problem using a Hybrid Evolutionary Hierarchical Reinforcement Learning (HE-HRL) approach, which combines a Genetic Algorithm for model placement with a Soft Actor-Critic agent for resource allocation. Experimental results reveal that EdgeCoInfer achieves a 100% task completion rate in high-concurrency scenarios, significantly reducing system costs by 76% and memory usage by 71.88% compared to existing methods.
Achieving 100% task completion in high-concurrency environments while slashing system costs by 76% and memory usage by nearly 72% could redefine edge computing for MLLMs.
Modern mobile applications predominantly execute concurrent Multimodal Large Language Models (MLLMs) to provide ubiquitous intelligence. However, satisfying this demand within edge environments faces significant challenges due to multi-task concurrency and strictly coupled hard constraints. To address these issues, we propose EdgeCoInfer, a framework enabling granularity-adaptive deployment by co-optimizing inter-model functional module sharing and \textbf{intra-model fine-grained partitioning}. We solve the underlying Mixed-Integer Non-Linear Programming (MINLP) problem via a Hybrid Evolutionary Hierarchical Reinforcement Learning (HE-HRL) paradigm, which synchronizes a Genetic Algorithm (GA) for discrete model placement with a Soft Actor-Critic (SAC) agent for continuous resource allocation. To navigate the sparse feasible region, we introduce a feasibility-guided constructive execution mechanism, integrating a constructive cut-step decoder with pre-act pruning and a two-phase curriculum strategy for stable adaptation. Experimental results demonstrate that EdgeCoInfer ensures a 100\% task completion rate in high-concurrency scenarios, achieving a 76\% reduction in system cost and 71.88\% memory savings compared to state-of-the-art baselines.