Search papers, labs, and topics across Lattice.
This study explores the impact of incorporating multimodal listening behaviors, specifically verbal backchannels and head nodding, into AI clones to enhance their authenticity and presence. By employing real-time prediction models, the researchers conducted a within-subjects study with 35 participants, revealing that these listening behaviors significantly improved perceptions of the avatar's attentiveness and the overall interaction experience. The findings suggest that creating more lifelike AI clones requires attention to interactive listening, not just vocal imitation and dialogue generation.
Enhancing AI clones with listening behaviors can dramatically elevate user perceptions of authenticity and engagement.
AI clones that imitate a specific person typically reproduce what the person says and how they sound, but not how they listen. We investigate whether adding multimodal listening behaviors gives such a clone more presence and authenticity. We integrated verbal backchannels and head nodding, driven by real-time prediction models, into an AI clone equipped with voice cloning and LLM-based responses. In a within-subjects study (N=35), adding these behaviors significantly improved the perceived attentiveness of the avatar, the sense of talking with the real person, and the feeling of co-presence. These results indicate that AI clone fidelity should extend beyond voice and response content to include interactive listening behavior.
