Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces GeoMER, a novel joint Euclidean-Hyperbolic geometry framework for multimodal emotion recognition (MER) that addresses the limitations of purely Euclidean models in capturing hierarchical emotional structures. By jointly modeling token-level cross-modal interactions in Euclidean space and abstract emotional representations in hyperbolic space via hyperbolic emotion-core routing, GeoMER effectively integrates fine-grained and hierarchical information. Experiments on the IEMOCAP benchmark demonstrate GeoMER's superior performance, achieving state-of-the-art weighted and unweighted accuracies.
Emotions' hierarchical structure is finally captured in multimodal recognition by fusing Euclidean token-level fusion with hyperbolic core routing, achieving state-of-the-art results.
Multimodal Emotion Recognition (MER) aims to understand human emotions by integrating information from multiple modalities such as speech and text. Psychological studies suggest that emotions exhibit latent hierarchical structured relationships in their semantic representations. However, most MER approaches rely on Euclidean representations, which capture fine-grained cross-modal interactions but do not explicitly provide a geometric prior for organizing hierarchical emotional structures. To mitigate this gap, we propose GeoMER, a joint Euclidean-Hyperbolic modeling framework that introduces hyperbolic emotion-core routing to capture higher-level, abstract emotional representations for MER. The proposed framework jointly models token-level cross-modal semantic interactions and latent emotion representations in two complementary geometric spaces. Specifically, GeoMER consists of a Euclidean Multimodal Fusion Module (EuclFusion) for multimodal representation fusion and a Hyperbolic Geometric Core Routing (HypGeoRoute) module that performs geometry-aware core assignment and aggregation in a hyperbolic manifold to capture more abstract feature representations. The HypGeoRoute module complements the Euclidean token-level modeling. We experiment on the IEMOCAP multimodal MER benchmark and demonstrate that the proposed method achieves the best performance, with 83.54% weighted accuracy and 83.35% unweighted accuracy, outperforming Euclidean baselines, including MemoCMT.