Search papers, labs, and topics across Lattice.
SignAgent, a novel agentic framework, leverages LLMs to address the bottleneck of manual linguistic annotation in sign language (SL) datasets. It uses a SignAgent Orchestrator, a reasoning LLM, to coordinate linguistic tools, and SignGraph, a knowledge-grounded LLM, to provide lexical and linguistic grounding. Evaluations on pseudo-gloss annotation and ID glossing tasks demonstrate the framework's strong performance in large-scale, linguistically-aware data annotation and curation.
Automating linguistically-grounded sign language annotation is now possible, unlocking scalable dataset curation previously limited by manual effort.
This paper introduces SignAgent, a novel agentic framework that utilises Large Language Models (LLMs) for scalable, linguistically-grounded Sign Language (SL) annotation and dataset curation. Traditional computational methods for SLs often operate at the gloss level, overlooking crucial linguistic nuances, while manual linguistic annotation remains a significant bottleneck, proving too slow and expensive for the creation of large-scale, phonologically-aware datasets. SignAgent addresses these challenges through SignAgent Orchestrator, a reasoning LLM that coordinates a suite of linguistic tools, and SignGraph, a knowledge-grounded LLM that provides lexical and linguistic grounding. We evaluate our framework on two downstream annotation tasks. First, on Pseudo-gloss Annotation, where the agent performs constrained assignment, using multi-modal evidence to extract and order suitable gloss labels for signed sequences. Second, on ID Glossing, where the agent detects and refines visual clusters by reasoning over both visual similarity and phonological overlap to correctly identify and group lexical sign variants. Our results demonstrate that our agentic approach achieves strong performance for large-scale, linguistically-aware data annotation and curation.
