Search papers, labs, and topics across Lattice.
This paper formulates financial advice generation as a reinforcement learning problem, utilizing Group Relative Policy Optimization (GRPO) to fine-tune an open-weight language model. The model is evaluated using a dual approach: an LLM-as-a-judge rubric for advice quality and a judge-independent audit based on a Conditional Average Treatment Effect (CATE) estimator. The results show that the GRPO-trained model achieves approximately double the gross-profit lift of the best commercial baseline while maintaining lower downside risk, highlighting the effectiveness of the method in generating actionable financial advice.
GRPO-trained models can deliver financial recommendations with twice the business value of leading commercial LLMs while minimizing risk.
Generating actionable financial advice from business records demands that models integrate numerical reasoning, domain knowledge, and sound judgment, while avoiding recommendations that could harm the business. Direct supervision is difficult: historical decisions are not necessarily optimal, and high-quality free-form labels are expensive to obtain. We formulate financial advice generation as a reinforcement learning problem and fine-tune an open-weight language model using Group Relative Policy Optimization (GRPO). Our reward is an LLM-as-a-judge rubric that scores each recommendation across multiple binary dimensions of advice quality, augmented with a safety gate for harm prevention. Since LLM-based evaluation alone cannot confirm whether improvements reflect genuine business value rather than adaptation to the judge, we complement it with a judge-independent audit based on a standard doubly-robust Conditional Average Treatment Effect (CATE) estimator. Under this observational off-policy audit, our trained LLM achieves approximately twice the estimated gross-profit lift of the strongest evaluated commercial baseline ($0.0228$ vs.\ $0.0104$), together with the lowest downside rate and the least negative tail risk of any policy evaluated. Notably, the two evaluations do not rank the baselines identically: the untrained base model places last on the judge rubric but second on the causal audit, indicating that the audit captures a signal the judge does not. Our results demonstrate that GRPO with a finance-grounded reward signal can produce substantially more useful business recommendations than commercial LLMs, and that a judge-independent causal audit is a valuable complement to, rather than a confirmation of, LLM-as-a-judge assessment in financial NLP.
