Search papers, labs, and topics across Lattice.
This paper introduces a multi-cluster co-inference framework for large AI models (LAIMs) that enhances inference efficiency in resource-constrained environments by coordinating multiple user clusters through an edge server with GPUs. By employing lightweight on-device LAIMs to capture diverse local features, the framework aggregates these features to improve inference accuracy while addressing the trade-offs between model pruning and collaborative performance. The authors develop a theoretical framework using rate-distortion theory to optimize model pruning ratios and resource allocation, resulting in significant improvements in accuracy and efficiency compared to existing benchmarks.
Achieving superior inference accuracy and resource efficiency in multi-cluster edge intelligence networks hinges on a delicate balance between model pruning and collaborative inference.
The increasing scale and computational demands of large artificial intelligence models (LAIMs) present significant challenges for efficient inference in resource-constrained distributed environments. In this paper, we propose a multi-cluster LAIM co-inference framework, where an edge server equipped with multiple graphics processing units (GPUs) coordinates multiple user clusters to execute inference tasks collaboratively. Within each cluster, devices capture data from diverse perspectives and employ lightweight on-device LAIMs to extract local features. These features are then transmitted to the edge server, where they are aggregated and fused to generate a more accurate inference outcome. To reveal the fundamental trade-off between model pruning and collaborative inference performance, we develop a theoretical framework that characterizes the impact of pruning ratios and device contributions using rate-distortion theory and partial information decomposition. Based on this analysis, we formulate a joint optimization problem that determines the model pruning ratio, the task scheduling strategy, the bandwidth allocation, and the transmission power, with the goal of minimizing the inference distortion while satisfying the constraints of latency, energy consumption, and server capacity. Extensive simulation results demonstrate that the proposed framework significantly outperforms existing benchmark schemes, achieving superior inference accuracy and resource efficiency in multi-cluster edge intelligence networks.
