Search papers, labs, and topics across Lattice.
This paper introduces $TCP_\alpha$, a novel confidence estimation method that addresses the overconfidence of deep neural networks in music information retrieval tasks by implementing a margin-controlled penalty for misclassified samples. The method ensures complete separation between confidence values of correct and incorrect predictions, thereby providing a reliable signal for users to trust predictions. Evaluation on tasks like raga identification and frame-wise ornamentation detection shows that $TCP_\alpha$ significantly enhances performance, achieving a macro-F1 score improvement from 0.89 to 0.98 by rejecting only the least-confident predictions.
$TCP_\alpha$ guarantees complete separation of confidence scores for correct and incorrect predictions, transforming how we trust model outputs in music information retrieval.
Deep neural networks are often overconfident, assigning high confidence even to incorrect predictions. Consequently, users lack a reliable signal for deciding when a prediction can be trusted. Post-hoc confidence estimation addresses this by training a lightweight auxiliary head over a frozen classifier. Existing targets, however, suffer from inherent ambiguity: they assign overlapping confidence values to correct and incorrect predictions, while errors near the decision boundary receive confidence scores indistinguishable from correct predictions. In this work, we propose $TCP_\alpha$, a novel confidence target that resolves these limitations by introducing a margin-controlled penalty for misclassified samples. We prove that $TCP_\alpha$ guarantees complete separation between the target values of correct and incorrect predictions, with a separation margin that is independent of the number of classes and increases monotonically with the penalty parameter. Since accurate classifiers naturally produce very few errors, learning these targets results in a severely imbalanced regression problem. We therefore present a systematic study of training strategies for learning under this imbalance and identify an effective training configuration through extensive ablation studies. We evaluate the proposed approach on r\=aga identification, investigate its robustness under domain shift, and further validate it on frame-wise ornamentation detection without modifying the selected configuration. Across all settings, $TCP_\alpha$ consistently outperforms existing confidence targets for failure prediction. Rejecting only the least-confident 8\% of predictions improves the base model's macro-F1 from 0.89 to 0.98, while fine-tuning the confidence head with only 5\% labeled samples from a new corpus effectively restores performance under domain shift.
