Search papers, labs, and topics across Lattice.
This paper introduces a framework for synthesizing a hierarchical benchmark for human action recognition, addressing the limitations of existing datasets that either lack temporal composition or are constrained by fixed hierarchies. By generating episodes from a flat single-label action corpus while maintaining real pre-extracted features, the authors create a four-level hierarchy that includes actions, activities, low-level intentions (LLIs), and high-level intentions (HLIs). The resulting benchmark, comprising 15,002 episodes, reveals a consistent compositional gap in performance across various model families, highlighting a structural challenge in the evaluation rather than a mere model deficiency.
A novel benchmark synthesis reveals a persistent compositional gap in human action recognition, challenging existing models to rethink their approach to hierarchical reasoning.
Recognizing human behavior across levels of abstraction, from atomic actions to long-horizon intentions, requires data annotated along a semantic hierarchy. Large corpora provide isolated, atomically labeled clips without temporal composition, whereas recorded composite-activity corpora offer shallow, domain-narrow, fixedhierarchies. A benchmark-generation and evaluation frameworkis proposed that synthesizes a four-level hierarchical-intention benchmark, spanning actions, activities, low-level intentions (LLIs), and high-level intentions (HLIs), from a flat single-label action corpus while retaining real pre-extracted features at the action level. Episodes are assembled by a transition model under a subject-consistency constraint, and a coverage-aware sampler reduces the subject usage Gini from 0.566 to 0.248. Synthesizing such a benchmark raises a circular-supervision risk that recorded datasets avoid: if the rules generating the episodes also govern the evaluation, models can succeed by recovering the generator rather than through genuine reasoning. Validity is addressed by design, holding sequence-generation rules disjoint from the first-order-logic rules used at evaluation. The instantiation yields 15,002 episodes. Four reference baselines from different model families characterize difficulty, not as recognition methods. A compositional held-out gap of 0.13 to 0.17 macro-F1 appears across all baselines, including a graph-aware model that recognizes best yet does not close the gap, indicating a structural property of the benchmark rather than a model artifact. A logic-free baseline still violates the held-out semantic rules above their intrinsic data rate, and the order-destroying control changes macro-F1 within seed variation, serving as a generator-consistency check. Theontology, transition model, and generator are released so the benchmark can beregenerated and extended.
