Search papers, labs, and topics across Lattice.
This paper introduces Spoken Function Calling (SFC), a new framework designed to improve spoken language understanding (SLU) for task-oriented dialogue systems by utilizing structured rule definitions. By curating an extensive suite of spoken functions and constructing the SFC-Bench dataset, the authors evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs) in semantic extraction tasks. The results indicate that SFC significantly enhances semantic extraction accuracy compared to traditional SLU methods, particularly in open-domain scenarios.
SFC redefines semantic understanding in spoken language tasks, achieving superior accuracy and adaptability in open-domain contexts.
Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.
DAMO