Search papers, labs, and topics across Lattice.
The paper addresses the problem of over-speaking and under-acting in tool-augmented LLM dialogue agents within the Commonsense Persona-grounded Dialogue Challenge (CPDC) 2025. They introduce a rule-based role prompting (RRP) approach incorporating character-card/scene-contract design and strict function call enforcement. RRP achieved a score of 0.571, outperforming a zero-shot baseline of 0.519, demonstrating its superiority over methods like automatic prompt optimization (APO).
Forget fancy prompt optimization: carefully crafted rule-based role prompts with enforced function calls dramatically improve the performance of tool-augmented LLM dialogue agents.
This report investigates approaches for prompting a tool-augmented large language model (LLM) to act as a role-playing dialogue agent in the API track of the Commonsense Persona-grounded Dialogue Challenge (CPDC) 2025. In this setting, dialogue agents often produce overly long in-character responses (over-speaking) while failing to use tools effectively according to the persona (under-acting), such as generating function calls that do not exist or making unnecessary tool calls before answering. We explore four prompting approaches to address these issues: 1) basic role prompting, 2) improved role prompting, 3) automatic prompt optimization (APO), and 4) rule-based role prompting. The rule-based role prompting (RRP) approach achieved the best performance through two novel techniques-character-card/scene-contract design and strict enforcement of function calling-which led to an overall score of 0.571, improving on the zero-shot baseline score of 0.519. These findings demonstrate that RRP design can substantially improve the effectiveness and reliability of role-playing dialogue agents compared with more elaborate methods such as APO. To support future efforts in developing persona prompts, we are open-sourcing all of our best-performing prompts and the APO tool Source code is available at https://github.com/scb-10x/apo
