Search papers, labs, and topics across Lattice.
Center for Research in Computer Vision, University of Central Florida;, N(EPi⋅LPi);Acce=Acc×ARS\text{ARS}=\frac{1}{N}\sum_{i=1}^{N}(\text{EP}_{i}\cdot\text{LP}_{i});\quad\text{Acc}_{e}=\text{Acc}\times\text{ARS}\vskip-3.0pt (9) where EP and LP represent early and late penalties. For each question, ProReady-QA provides a ground-truth evidence window [ts,te][t_{s},t_{e}], where tst_{s} marks when sufficient evidence first appears and tet_{e} when it ceases to be valid. Given a model answers at time tat_{a}, these intervals enable ARS to evaluate timing behavior through complementary early and late penalties aligned with our readiness-aware formulation. Early Penalty (EP). Answering before any supporting evidence appears is the most severe readiness failure, as it reflects speculation rather than observation. To discourage such behavior, the Early Penalty sharply decreases as the model answers earlier than the evidence onset tst_{s}, scaled by the median evidence duration τ\tau for consistency. EP=softmin(1,2σ(γeta−tsτ+ϵ))\text{EP}=\text{softmin}\left(1,2\ \sigma\left(\gamma_{e}\dfrac{t_{a}-t_{s}}{\tau+\epsilon}\right)\right) (10) Here, σ(⋅)\sigma(\cdot) is sigmoid, γe\gamma_{e} controls penalty sharpness, ϵ\epsilon ensures numerical stability. EP→1\text{EP}\to 1 as tat_{a} approaches tst_{s}, while early answers yield lower scores. Late Penalty (LP). After the evidence ends at tet_{e}, delayed responses indicate hesitation rather than hallucination. To capture this milder form of readiness error, the Late Penalty gently decreases with delay, encouraging timely responses without overly penalizing slight delays: LP=softmin(1,softmax(0,1−γℓta−teτ+ϵ))\text{LP}=\text{softmin}\left(1,\text{softmax}\left(0,1-\gamma_{\ell}\dfrac{t_{a}-t_{e}}{\tau+\epsilon}\right)\right) (11) where γℓ\gamma_{\ell} controls decay slope, keeping LP near 1 for minor delays and lower for prolonged ones. For questions with multiple valid answers, we compute ARS separately for each turn and report their average. 5 Experiments Table 3: Performance comparison of streaming video understanding.
1
0
2
Answering at the wrong time can be as bad as answering incorrectly in streaming video, so this work introduces a new framework that learns when to answer based on the availability of supporting visual evidence.