Search papers, labs, and topics across Lattice.
This paper introduces a unified framework for the customization and deployment of multi-agent systems based on large language models, addressing the challenges of domain-specific adaptation and high inference costs. The framework consists of two stages: Agentic Model Customization, which employs continual pretraining, supervised fine-tuning, and preference optimization, and Inference Optimization, which utilizes speculative decoding and FP8 quantization for efficient serving. The proposed approach achieves a 4.48x speedup in throughput across enterprise workloads while maintaining performance and enhancing robustness in long-tail scenarios.
Achieving a 4.48x speedup in throughput for multi-agent systems without sacrificing performance could revolutionize enterprise applications.
Large language model (LLM)-based multi-agent systems demonstrate strong performance on complex reasoning and task execution, enabling broad enterprise applications. However, production deployment remains challenging due to domain-specific customization requirements and high latency and inference costs in agentic workflows. We propose a unified framework for customization and efficient deployment of multi-agent systems in real-world settings. The first stage, Agentic Model Customization, combines continual pretraining, supervised fine-tuning, and preference optimization to adapt a compact model to specialized domains while retaining strong agentic capabilities. The second stage, Inference Optimization, integrates speculative decoding and FP8 quantization with targeted calibration to enable cost-efficient serving with minimal quality loss. Across enterprise workloads, our framework enables rapid domain adaptation and achieves a 4.48x speedup in throughput while maintaining performance and improving robustness on long-tail scenarios.
