Search papers, labs, and topics across Lattice.
This paper introduces an end-to-end RLAIF framework designed to generate portable job search queries that abstract seeker-specific identifiers while maintaining generalizable qualifications. The authors reveal that robust reward shaping is crucial for optimizing performance in environments where LLMs serve as judges, as traditional algorithms can be easily exploited by adversarial reward signals. Their findings indicate that implementing a deterministic reward floor significantly improves query quality, demonstrating a 2.4x performance gain when reward-shaping disciplines are enforced during training.
Reward shaping can be the difference between success and failure in optimizing job search queries, with a simple rule-based adjustment yielding a 2.4x performance boost.
Job-search platforms rely on low-bandwidth query interfaces that often fail to capture the high-dimensional complexity of candidate profiles. We present an end-to-end RLAIF (Reinforcement Learning from AI Feedback) framework to generate \emph{portable} job search queries, terms that abstract away seeker-specific identifiers while preserving generalizable qualifications. This task introduces a highly adversarial reward surface where policy optimization frequently exploits flaws in LLM-as-judge rubrics, resulting in degenerate verbatim-copying behaviors. We conducted comprehensive empirical experiments to isolate the impact of optimization mechanics against structured reward engineering. Our results demonstrate that for critic-free optimizers, performance is overwhelmingly dictated by robust reward shaping, rendering the specific choice of algorithm largely immaterial. While critic-free per-rollout baseline methods (RLOO and REINFORCE++) natively resist reward-hacking, the group-relative advantage normalization in GRPO appears uniquely sensitive to spurious reward signals, making it disproportionately susceptible to exploitation. We show that introducing a deterministic, rule-based reward floor to correct for rewards assigned to verbatim copying mitigates this failure mode, resulting in a substantial $+0.147$ quality improvement on a cross-family evaluation judge. Ultimately, we show that the training-time reward model inflates performance gains by $2.4\times$, confirming that the training success is fundamentally dependent on enforcing reward-shaping disciplines rather than selecting alternative optimizers.
