Search papers, labs, and topics across Lattice.
This paper addresses the challenge of creating large, aligned cross-modal datasets for music-flavor research by investigating the transferability of audio-flavor correlations from a small, annotated dataset to a larger, synthetically labeled dataset. They validate computationally derived flavor targets against human perception through online listening tests, demonstrating significant alignment. The key result is the confirmation that cross-modal structure is preserved across supervision regimes, supporting the presence of sonic seasoning effects in synthetic FMA annotations.
Synthetically labeled music data can capture real human perceptions of flavor, unlocking new possibilities for large-scale cross-modal AI research.
Collecting large, aligned cross-modal datasets for music-flavor research is difficult because perceptual experiments are costly and small by design. We address this bottleneck through two complementary experiments. The first tests whether audio-flavor correlations, feature-importance rankings, and latent-factor structure transfer from an experimental soundtracks collection (257~tracks with human annotations) to a large FMA-derived corpus ($\sim$49,300 segments with synthetic labels). The second validates computational flavor targets -- derived from food chemistry via a reproducible pipeline -- against human perception in an online listener study (49~participants, 20~tracks). Results from both experiments converge: the quantitative transfer analysis confirms that cross-modal structure is preserved across supervision regimes, and the perceptual evaluation shows significant alignment between computational targets and listener ratings (permutation $p<0.0001$, Mantel $r=0.45$, Procrustes $m^2=0.51$). Together, these findings support the conclusion that sonic seasoning effects are present in synthetic FMA annotations. We release datasets and companion code to support reproducible cross-modal AI research.
