Ramani Duraiswami

VIBE achieves superior instruction adherence and controllability in music generation from video, setting a new standard for semantic alignment in multimodal tasks.

TEMPO not only achieves superior timestamping accuracy in audio-language models but also integrates innovative techniques like atomic timestamp tokens and reinforcement learning for enhanced performance.

AV-Flamingo outperforms existing models on complex audio-visual tasks, revealing that size isn't everything when it comes to reasoning capabilities.

GAIA achieves a 64% reduction in error for airfoil flow reconstruction, setting a new benchmark for operator learning in complex geometries.

LALMs can boost their temporal reasoning accuracy by 3.2% simply by better redistributing attention across audio tokens rather than relying on textual cues.

Generate semantically aligned, high-fidelity music for videos with unprecedented speed and control by combining autoregressive planning and diffusion.

Audio-language models can now reason about 30-minute-long audio clips with timestamp-grounded intermediate steps, unlocking a new level of fine-grained understanding.

Current multimodal models are surprisingly bad at understanding long, complex videos, struggling to integrate audio, visual, and text cues even for basic reasoning tasks.

Forget HRTFs: a differentiable multi-sphere scattering model inspired by underwater animal acoustics offers a new foundation for spatial audio processing and localization.