Search papers, labs, and topics across Lattice.
This paper introduces an efficient framework for financial sentiment analysis that leverages distillation with synthetic data to enhance compact models in low-resource settings. By clustering a small set of hand-labeled examples to generate more representative synthetic data, the approach enables these models to outperform larger teacher models, particularly in complex and noisy text domains. The findings highlight a significant advancement in resource-efficient domain adaptation for financial NLP, reducing the reliance on extensive human labeling efforts.
Compact models can surpass larger instruction-tuned models in financial sentiment analysis by leveraging intelligently generated synthetic data.
Large instruction-following models are powerful but costly to deploy, particularly in finance, where labelled data are limited by confidentiality and expert annotation cost. We present an efficient framework for financial sentiment analysis through distillation with synthetic data, transferring knowledge from a large instruction-tuned teacher to compact student models. The framework is designed for low-resource conditions, where a small set of real examples are collected and labelled by hand. The framework then clusters the examples and uses the clusters to select seeds for generating synthetic examples via structured few-shot prompting. Experiments show that clustering-based seed selection yields more representative synthetic data than random sampling, enabling compact models to achieve strong performance with minimal supervision. Notably, on a more complex and noisy text domain, the compact model trained on the complete synthetic-seed corpus even outperforms the teacher model, while remaining competitive on formal text. The framework provides a practical route toward resource-efficient domain adaptation in financial NLP with minimal human labelling effort.
