Search papers, labs, and topics across Lattice.
This paper introduces the Advanced Acceptance Score (AAS), a novel holistic measure for evaluating biometric quantification in hand gestures by considering ranking order, relevance of output scores, and identity feature disentanglement. AAS rewards higher scores for high-ranked gestures and lower scores for low-ranked gestures, compensates for trends between output and ground truth scores, and discounts for identity feature entanglement. Experiments across three datasets and five state-of-the-art models demonstrate that AAS selects more appropriate optimal scores compared to existing measures and exhibits correlation with existing measures, validating its reliability.
Stop relying on error rates to evaluate biometric hand gesture quantification: a new Advanced Acceptance Score (AAS) better captures the goodness of these scores.
Quantifying biometric characteristics within hand gestures involve derivation of fitness scores from a gesture and identity aware feature space. However, evaluating the quality of these scores remains an open question. Existing biometric capacity estimation literature relies upon error rates. But these rates do not indicate goodness of scores. Thus, in this manuscript we present an exhaustive set of evaluation measures. We firstly identify ranking order and relevance of output scores as the primary basis for evaluation. In particular, we consider both rank deviation as well as rewards for: (i) higher scores of high ranked gestures and (ii) lower scores of low ranked gestures. We also compensate for correspondence between trends of output and ground truth scores. Finally, we account for disentanglement between identity features of gestures as a discounting factor. Integrating these elements with adequate weighting, we formulate advanced acceptance score as a holistic evaluation measure. To assess effectivity of the proposed we perform in-depth experimentation over three datasets with five state-of-the-art (SOTA) models. Results show that the optimal score selected with our measure is more appropriate than existing other measures. Also, our proposed measure depicts correlation with existing measures. This further validates its reliability. We have made our \href{https://github.com/AmanVerma2307/MeasureSuite}{code} public.
