Search papers, labs, and topics across Lattice.
This paper introduces ear-VAE2, a complex-spectral autoencoder that addresses common failure modes in high-compression audio reconstruction, specifically high-frequency loss, phase incoherence, and stereo-image collapse. By leveraging a frequency-aware approach and implementing the Duplex-Aware Refiner, the model significantly improves reconstruction quality, achieving superior performance on multiple metrics compared to existing methods. Notably, ear-VAE2 reduces Mel Distance by 19.4% while using approximately 45% fewer residual-output dimensions, demonstrating its efficiency and effectiveness in high-fidelity music reconstruction.
Achieving a 19.4% reduction in Mel Distance while using 45% fewer parameters, ear-VAE2 redefines the standards for high-fidelity music reconstruction.
Continuous-latent audio autoencoders form the backbone of latent music generators, yet decoders at high compression rates commonly exhibit three failure modes: high-frequency loss, phase incoherence, and stereo-image collapse. These share a structural root: waveform autoencoders lack an explicit frequency axis, leaving no handle for targeted per-band correction. Among five matched-budget representations, the complex STFT achieves the lowest full-band and high-frequency spectral distances, providing direct access to magnitude and phase at every bin. Building on this, we present ear-VAE2, a complex-spectral autoencoder with cross-channel interaction. Spec-SnakeBeta learns a periodic activation per frequency bin with frequency-dependent initialization, outperforming other activation variants while using fewer parameters than the fully independent variant. Duplex-Aware Refiner applies band-specific corrections to magnitude and phase following duplex theory of sound localization. On the 546-track Song Describer Dataset, ear-VAE2 achieves the best point estimates on five of seven reconstruction metrics. The Duplex-Aware Refiner reduces Mel Distance by 19.4% and uses ~45% fewer residual-output dimensions than the Unconstrained Refiner, while also lowering spectral distances, spatial-cue errors, and receiving higher ratings from professional engineers. The downstream generator using ear-VAE2 latents achieves better point estimates on all 12 automatic metrics.Demo page is available at https://eps-acoustic-revolution-lab.github.io/EAR_VAE2/.
