Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a rubric-guided SpeechLLM designed for automated L2 speech assessment that provides both proficiency labels and natural-language rationales. By employing a hybrid training approach that combines supervised fine-tuning with Bounded Direct Preference Optimization, the model assesses multiple aspects of speech, including accuracy, fluency, and prosody, at both sentence and word/phoneme levels. The results on the SpeechOcean762 dataset show that this multi-granular model not only matches but often surpasses the performance of traditional single-granularity models, while also offering insights into the reliability of its generated rationales.
Natural-language rationales from a SpeechLLM reveal surprising inconsistencies in faithfulness at the word level, challenging assumptions about interpretability in automated assessments.
Automated L2 speech assessment can assign proficiency labels, but often lacks interpretability. We propose a rubric-guided SpeechLLM for multi-aspect, multi-granular assessment, trained with a hybrid objective combining supervised fine-tuning and Bounded Direct Preference Optimization. The model jointly predicts ordinal labels at the sentence-level (accuracy, fluency, prosody), word/phoneme-level accuracy, and generates a natural-language rationale in the same response. On SpeechOcean762, our approach matches or outperforms single-granularity models while remaining competitive with prior approaches. We analyze rationale reliability along two axes: self-consistency with model predictions and alignment with ground-truth labels, using sentiment consistency (plausibility) and mention-based agreement (faithfulness). Rationales are plausible at the sentence level, but faithfulness degrades at the word/phoneme level: references are sparse and weakly aligned with token-level labels.