Search papers, labs, and topics across Lattice.
This study investigates the effectiveness of domain adaptation in sentiment analysis using frozen pre-trained language model (PLM) backbones, specifically examining various embedding sizes and architectures. The authors find that while domain adaptation yields negligible improvements on movie reviews, it significantly enhances performance for smaller general-purpose models on financial news, revealing a nuanced relationship between backbone specialization and adaptation methods. Notably, the use of adversarial alignment (DANN) can degrade performance for specialized models like FinBERT, suggesting that pre-existing domain knowledge can be compromised through inappropriate adaptation techniques.
Domain adaptation can backfire, eroding performance in specialized models while boosting smaller backbones in specific contexts.
Sentiment analysis with frozen pre-trained language model (PLM) backbones has become a common paradigm, yet the practical benefit of explicit domain adaptation remains unclear, particularly when backbones encode varying degrees of target-domain knowledge. We present a preliminary case study evaluating a controlled family of frozen embedding backbones (Qwen3-Embedding 0.6B, 4B, 8B), alongside RoBERTa-base and FinBERT. We train a lightweight MLP adapter on consumer reviews using Domain-Adversarial Neural Networks (DANN), Maximum Mean Discrepancy (MMD), and Supervised Contrastive Learning (SCL), and evaluate transfer to movie reviews (SST-2) and a heavily restricted subset of financial news (Financial PhraseBank). Within this constrained sample, we observe two distinct transfer patterns. On SST-2, domain adaptation provides negligible gain regardless of scale. On the financial subset, explicit domain adaptation appears to recover substantial performance for small general-purpose backbones. Notably, we find that adversarial alignment (DANN) is associated with degraded performance for domain-specialized backbones like FinBERT, consistent with erosion of pre-existing domain-specific structure, whereas supervised contrastive loss appears to preserve it. These preliminary findings suggest that the efficacy of explicit domain adaptation is highly contingent on whether the frozen backbone already possesses target-domain coverage.
