Search papers, labs, and topics across Lattice.
This paper investigates the issue of Argument Language Mismatch (ALM) in multilingual settings for Large Language Models (LLMs) when calling APIs, where models generate argument values in inconsistent languages despite selecting the correct tool. The authors find that supervised fine-tuning (SFT) significantly enhances argument language consistency and overall function call accuracy, achieving performance on par with or exceeding more complex reinforcement learning (RL) methods. Additionally, while RL approaches like Group Relative Policy Optimization (GRPO) offer some incremental improvements, they primarily enhance generalization rather than fundamentally altering performance outcomes.
Supervised fine-tuning outperforms complex reinforcement learning techniques in ensuring multilingual API reliability, challenging the notion that more sophisticated methods are always necessary.
The reliability of Large Language Models (LLMs) for API calling degrades in multilingual settings. A common failure occurs when a model selects the correct tool but generates argument values in an inconsistent language, which we term Argument Language Mismatch (ALM). Although semantically correct, such outputs are operationally invalid and not captured by standard API-calling metrics. We revisit post-training strategies for mitigating ALM and find that, in our benchmark, supervised fine-tuning (SFT) provides a strong baseline, substantially improving argument language consistency and end-to-end function call accuracy. Under consistent model selection, SFT achieves performance comparable to, and sometimes exceeding more complex reinforcement learning (RL) approaches. We further examine whether RL with structured, argument-aware rewards offers additional benefits. While methods such as Group Relative Policy Optimization (GRPO) can improve language consistency and better preserve general reasoning ability, these gains are incremental and most pronounced in generalization and multi-objective trade-offs. Overall, our results suggest that much of the performance in multilingual API grounding can be achieved through careful supervised training, with RL providing targeted rather than fundamental improvements.
