Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Earnings25 is a novel benchmark designed to evaluate automatic speech recognition (ASR) systems specifically for English-language earnings calls, consisting of 498 hours of full calls and 46 hours of segmented data. This benchmark includes detailed transcripts and metadata, allowing for nuanced assessments that go beyond traditional aggregate word error rates (WER) by considering speaker roles and industry contexts. The authors provide reproducible baseline performance metrics for leading ASR models, Whisper and Parakeet-TDT, highlighting the benchmark's utility in real-world financial settings.
Earnings25 reveals that traditional ASR evaluation metrics fall short in finance, necessitating a more nuanced approach that considers speaker and industry context.
We introduce Earnings25, a finance-domain benchmark for evaluating automatic speech recognition (ASR) on English-language earnings calls under realistic conditions. Earnings25 comprises two complementary test sets: (i) testset-full, 498 hours of full English-language S&P 500 earnings calls from Q4 2025, and (ii) testset-segmented, a 46-hour industry-balanced set of 290 segments sampled from English-language U.S. earnings calls in 2025. The benchmark provides aligned transcripts and structured metadata, including speaker roles, industry labels, and call structure, enabling speaker- and industry-aware evaluation beyond aggregate word error rate (WER). We report reproducible baselines for Whisper and Parakeet-TDT using standardized scoring.