Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces CAMO, a novel ensemble method designed to address class imbalance in language model categorization tasks by dynamically boosting underrepresented classes. CAMO leverages a hierarchical procedure incorporating vote distributions, confidence calibration, and inter-model uncertainty to improve minority class performance. Experiments on highly imbalanced datasets (DIAR-AI/Emotion and BEA 2025) demonstrate that CAMO consistently achieves the highest strict macro F1-score compared to seven other ensemble methods across various language models (LLMs and SLMs) in both zero-shot and fine-tuned settings.
Class imbalance cripples language model evaluation, but CAMO, a new ensemble method, dynamically boosts minority classes to achieve state-of-the-art macro F1-scores on imbalanced datasets.
Real-world categorization is severely hampered by class imbalance because traditional ensembles favor majority classes, which lowers minority performance and overall F1-score. We provide a unique ensemble technique for imbalanced problems called CAMO (Class-Aware Minority-Optimized).Through a hierarchical procedure that incorporates vote distributions, confidence calibration, and inter model uncertainty, CAMO dynamically boosts underrepresented classes while preserving and amplifying minority forecasts.We verify CAMO on two highly unbalanced, domain-specific benchmarks: the DIAR-AI/Emotion dataset and the ternary BEA 2025 dataset. We benchmark against seven proven ensemble algorithms using eight different language models (three LLMs and five SLMs) under zero-shot and fine-tuned settings .With refined models, CAMO consistently earns the greatest strict macro F1-score, setting a new benchmark. Its benefit works in concert with model adaptation, showing that the best ensemble choice depends on model properties .This proves that CAMO is a reliable, domain-neutral framework for unbalanced categorization.