Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study explores the effectiveness of a single lightweight neural reranker for action selection across multiple diverse environments, aiming to reduce the inference costs associated with large language model agents. By jointly training DeBERTa-v3 on ALFWorld, WebShop, and ScienceWorld, the authors demonstrate that cross-environment training significantly enhances performance, achieving a mean net gain of +0.551 while maintaining competitive results in individual environments. The findings reveal that sample-efficient adaptation can be achieved with minimal target-domain data, emphasizing the importance of data diversity over model capacity in improving performance.
A single neural reranker can outperform specialized models in diverse environments while being highly sample-efficient, achieving 93% of full performance with just 9.2% of the target-domain data.
Large language model agents achieve strong performance on text-based benchmarks but incur prohibitive inference costs, motivating the use of compact neural rerankers for action selection. We investigate whether a single lightweight model can perform action selection across multiple diverse environments, a capability that would eliminate per-environment model maintenance. Training DeBERTa-v3 (184M-434M parameters) jointly on ALFWorld, WebShop, and ScienceWorld with minority-class upsampling, we find that rebalanced two-environment joint training substantially improves over single-environment ALFWorld performance (net gain +0.412) while maintaining competitive WebShop performance (+0.214 vs. +0.249 single-environment). Three-environment training yields a mean combined net gain of +0.551 +/- 0.024 across 4 seeds, with per-environment results approaching specialized single-environment models while providing positive cross-domain transfer. Cross-environment adaptation is highly sample-efficient: fine-tuning on only 9.2% of target-domain data recovers 93% of full-data performance, and scaling model capacity yields limited benefits, indicating data diversity is the primary driver. Environment-aware LoRA adapter routing with PCGrad achieves a best-seed result of +0.611 (seed 42), with seeds 456 and 789 at +0.554 and +0.559, but exhibits high variance due to seed 123 collapsing to +0.263 (4-seed mean +0.497 +/- 0.158), representing a promising but currently unstable direction. Joint training with clean splits and data rebalancing is a key ingredient. We will release our three-environment benchmark of 51,580 training instances (41,740 raw unique states with minority-class upsampling) and all model checkpoints upon acceptance.