Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SocialReasonBench, a novel video multiple-choice QA benchmark designed to evaluate social reasoning in complex, interactive narrative scenarios derived from the game Detroit: Become Human. By utilizing branching storylines that reflect player decisions and their social consequences, the benchmark assesses seven dimensions of reasoning, including intent recognition and counterfactual reasoning. Experiments reveal that while large multimodal models can handle basic social understanding, they struggle significantly with more complex reasoning tasks, indicating a critical gap in their ability to interpret nuanced social dynamics.
Large multimodal models excel at basic social understanding but falter dramatically when faced with counterfactual reasoning and causal inference in complex narratives.
Recent advances in Large Multimodal Models (LMMs) have greatly improved video understanding, yet their ability to reason about human-centered social situations remains limited. Existing benchmarks typically rely on videos with a single observed trajectory, making it difficult to determine whether models truly understand social dynamics or merely exploit recurring narrative patterns. We introduce SocialReasonBench, a video multiple-choice QA benchmark for evaluating socially grounded reasoning in scenarios derived from interactive narratives. Built from gameplay videos of Detroit: Become Human, the benchmark leverages branching storylines where player decisions lead to alternative social outcomes that can be checked against the game's own script, flowchart, and recorded branches. We develop a multi-agent curation pipeline that localizes socially meaningful clips, grounds answer labels in game-state signals, and generates theory-guided questions with diagnostic distractors. SocialReasonBench covers seven reasoning dimensions, including intent recognition, emotional empathy, moral dilemma, counterfactual reasoning, and causal antecedent. Experiments on contemporary LMMs show that models perform reasonably well on basic social understanding but struggle with counterfactual and causal reasoning. Further ablation and diagnostic error analyses reveal that models often depend on incomplete modality cues and fall into reasoning traps such as visual shortcuts, highlighting a gap between observable event recognition and deeper reasoning over latent social states.