Rong Qu

MLLMs still struggle with the spatiotemporal reasoning needed to understand surgical videos, even with chain-of-thought prompting.

MLLMs still struggle to integrate diverse data for clinical reasoning, as evidenced by their poor performance on a new ophthalmology benchmark spanning image quality assessment to diagnosis.

Finally, realistic and diverse listener reactions to speech can be automatically generated, moving beyond simple retrieval or LLM-driven approaches.