Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the MediaGraph Data Model and its implementation, MeGraS, which integrates multimedia content as nodes within a multimodal knowledge graph, overcoming the limitations of traditional content-agnostic approaches. By allowing direct access to a document's intrinsic content, the framework facilitates advanced querying capabilities such as feature-based similarity search and dynamic segmentation. The extension of the SPARQL query language within MeGraS marks a significant advancement in the management and analysis of multimodal data, paving the way for more sophisticated knowledge graph applications.
Integrating multimedia content directly into knowledge graphs transforms how we query and analyze complex relationships across diverse media types.
Multimodal knowledge graphs typically treat multimedia documents as opaque, external entities. This content-agnostic approach constrains retrieval and analysis by isolating media from the graph's core structure, hindering the ability to capture and query complex relationships across media types. To address this, we introduce the MediaGraph Data Model and its prototypical implementation MeGraS, the MediaGraph Store, a novel approach that integrates multimedia content as graph nodes. This paradigm shift enables the query engine to directly access and process a document's intrinsic content, allowing for native operations such as feature-based similarity search, dynamic segmentation, and the inference of non-materialized relations. By extending the SPARQL query language, MeGraS provides a cohesive platform for the storage, management, and expressive querying of multimodal data. MeGraS is open-source software that establishes a new framework, moving the field toward content-aware multimodal knowledge graphs.