Search papers, labs, and topics across Lattice.
This paper introduces an evidence-grounded multimodal pipeline designed to construct knowledge graphs from lecture videos by integrating transcriptions, optical character recognition (OCR), and visual evidence. The approach effectively retains and validates concepts and relationships, resulting in a provenance-rich knowledge graph that achieved high accuracy in a preliminary retrieval test. Key findings include the extraction of 1,022 concept mentions and 312 relationships, with 100% accuracy in retrieving relevant information from the constructed graph.
Achieving 100% retrieval accuracy, this multimodal pipeline transforms lecture videos into a rich, auditable knowledge graph that captures the full spectrum of educational content.
Lecture videos distribute knowledge across speech, slide text, diagrams, equations, and presentation order, which transcript-only retrieval does not fully preserve. This paper presents an evidence-grounded multimodal pipeline that transcribes lectures, selects semantic anchors, applies optical character recognition (OCR), and uses a vision-language model to extract only concepts and typed relationships supported by transcript, OCR, or visual evidence. Mentions are validated and canonicalized into a provenance-rich knowledge graph. On three neural-network lectures, the pipeline processed 3,118 frames, 756 transcript segments, and 559 anchors. It retained 1,022 concept and 312 relationship mentions, yielding 172 canonical concepts and 282 relationships with 90.38% endpoint coverage. A preliminary three question retrieval test achieved 100% top-1 and top-3 accuracy and 100% mean top-5 recall. The contribution is an auditable construction method rather than a state-of-the-art performance claim.
