Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Profy, a weakly supervised system designed to enhance piano practice by providing interpretable, time-aligned feedback based on expert and amateur performance ratings. By analyzing synchronized key-motion and audio data from 73 pianists, Profy generates clip-level predictions that highlight specific passages for review, enabling learners to focus on areas needing improvement. The system's effectiveness is evidenced by a strong correlation (Pearson r=0.61) between its highlight scores and expert annotations, demonstrating its potential to transform practice feedback from summary-based to actionable insights.
Profy reveals that nuanced, time-aligned feedback can significantly enhance piano practice, aligning learner focus with expert insights.
The quality of piano performance depends on nuanced timing, articulation, and dynamic control, but practice feedback is often summary-based and hard to act on. We introduce Profy, a weakly supervised system that learns from take-level labels derived from aggregated listener ratings (expert-labeled vs. amateur-labeled) to produce time-aligned highlights for review during piano practice. We collected synchronized 1 kHz key-motion and audio from 73 pianists and used 1,083 valid takes for modeling and evaluation. The model outputs clip-level predictions together with evidence scores on a shared resampled model time base for visualization. On 20 amateur clips from short technique studies annotated by 21 expert pianists, the displayed highlight score aligns with passages that expert pianists marked for review despite training without localized labels (Pearson r=0.61, ROC-AUC 0.75). Rather than summarizing a take with a single global score, Profy helps learners decide where to inspect next by supporting scrubbing, looping, and focused replay of time-localized passages associated with expert-amateur differences.