Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study explores the effectiveness of orchestrated ensembles of small language models (SLMs) in malware analysis, particularly in interpreting complex detonation reports. By evaluating various orchestration architectures, including multi-agent pipelines and adversarial debate frameworks, the authors found that a hybrid system combining evidence-grounded pipelines with adversarial reasoning outperformed both specialized cyber models and leading frontier LLMs. The best-performing hybrid architecture achieved an overall accuracy of 35.30%, demonstrating that smaller, open-weight models can be effectively utilized in resource-constrained environments without sacrificing performance.
Orchestrating small language models can surpass the performance of single large models in malware analysis, achieving significant accuracy improvements with fewer resources.
Malware analysis demands rapid interpretation of complex detonation reports spanning filesystem, network, and process behaviours. While large language models (LLMs) demonstrate impressive capabilities for technical artifact interpretation, the opacity and escalating API costs of closed-weight frontier models motivate exploration of open-weight alternatives. However, many open-weight models are large, demanding significant compute resources and incurring non-trivial hosting costs that place them beyond reach for resource-constrained deployments. This paper investigates whether orchestrated ensembles of small language models (SLMs) can match or exceed single LLM performance on structured questions about malware detonation reports. We established baselines by testing eleven open-weight SLMs, three cyber security pre-trained models, and six frontier LLMs on Meta's CyberSecEval Malware Analysis benchmark. We then designed and evaluated four orchestration architectures: (i) a multi-agent pipeline that decomposes analysis into structured evidence-collection and reasoning stages, (ii) an adversarial debate framework in which two agents iteratively critique each other's reasoning, (iii) a hierarchical consultation system that pairs a general-purpose SLM with a cyber-specialised expert model, and (iv) a hybrid architecture that combines evidence-grounded pipelines with adversarial debate reasoning. The hybrid system (Qwen3-4B with Foundation-Sec-8B) achieved 35.30% overall accuracy, exceeding the strongest cyber-specialised baseline (22.54%) and the strongest ungrounded frontier baseline (34.77%); when given the same evidence pipeline, grounded Gemini remained the strongest configuration at 38.22%. These findings show that evidence-grounded orchestration can substantially improve the performance of collaborative SLMs for supporting interpretation of malware detonation reports.