Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SocialLDG, a multi-task learning framework that enables robots to infer user internal states and predict future behavior by modeling the dynamic relationship between latent states and observable actions. The framework uses a language model to introduce lexical priors for each task and employs dynamic graph learning to model task affinity evolving with time. SocialLDG achieves state-of-the-art performance on human-robot interaction datasets, supports task scalability, and offers insights into the dynamics of human decision-making.
Robots can now better understand and predict human behavior by explicitly modeling the dynamic relationships between internal states and observable actions, leading to more natural and effective social interactions.
For a robot to be called socially intelligent, it must be able to infer users internal states from their current behaviour, predict the users future behaviour, and if required, respond appropriately. In this work, we investigate how robots can be endowed with such social intelligence by modelling the dynamic relationship between user's internal states (latent) and actions (observable state). Our premise is that these states arise from the same underlying socio-cognitive process and influence each other dynamically. Drawing inspiration from theories in Cognitive Science, we propose a novel multi-task learning framework, termed as \textbf{SocialLDG} that explicitly models the dynamic relationship among the states represent as six distinct tasks. Our framework uses a language model to introduce lexical priors for each task and employs dynamic graph learning to model task affinity evolving with time. SocialLDG has three advantages: First, it achieves state-of-the-art performance on two challenging human-robot social interaction datasets available publicly. Second, it supports strong task scalability by learning new tasks seamlessly without catastrophic forgetting. Finally, benefiting from explicit modelling task affinity, it offers insights on how different interactions unfolds in time and how the internal states and observable actions influence each other in human decision making.