Question Answering for Diagram-Rich Technical Meeting Videos

2026-07-11

AI Summary

This work addresses the challenge of efficiently and accurately extracting key knowledge from technical conference videos, where critical information is dispersed across speech, slides, and diagrams such as UML charts. To tackle this, the authors propose LMVQA, the first system enabling efficient, traceable multimodal question answering specifically designed for diagram-intensive conference videos. LMVQA processes videos in a single pass to construct a timestamped, reusable evidence repository that integrates audio, visual, and diagram semantics to support precise answer generation. Evaluated on both the Ciena dataset and public benchmarks, the system boosts answer accuracy from 31% to 94% and from 21% to 88%, respectively, while reducing response latency from 81.3 seconds and 98.4 seconds to 3.3 seconds and 9.2 seconds, and cutting LLM invocation costs by approximately 75%.

Abstract

Software engineering increasingly relies on asynchronous communication artifacts, including recorded meetings where stakeholders discuss concerns, rationale, and decisions. These meetings often include diagram-based representations of requirements, system behavior, component interactions, and trace dependencies. Accessing knowledge from these meetings is challenging because recordings are long and relevant evidence is distributed across speech, slides, and technical diagrams. This paper reports our industrial experience developing and evaluating LMVQA, an LLM-based multimodal question-answering system for technical meeting videos. Developed in collaboration with engineers at Ciena, LMVQA supports the understanding of requirements and design intent by grounding answers in audio and visual evidence, with explicit handling of diagram-rich content such as requirements and UML diagrams. It processes each video once to build a reusable time-stamped evidence corpus for grounded question answering. Across a Ciena dataset and a public dataset, we show that LMVQA significantly improves answer accuracy compared to a state-of-the-art baseline, from 31% to 94% on the Ciena dataset and from 21% to 88% on the public dataset, with larger gains on diagram-rich videos. We further show that, after one-time indexing, LMVQA reduces average response time from 81.3s to 3.3s on Ciena and from 98.4s to 9.2s on the public dataset, while lowering average token-based LLM API cost by about 75%. Finally, our interviews with three domain experts show that engineers particularly value LMVQA for locating software-engineering-relevant information, revisiting rationale, and tracing answers to specific video segments.

Problem

• question answering
• technical meeting videos
• diagram-rich content
• multimodal retrieval
• software engineering knowledge

Innovation

• multimodal question answering
• diagram understanding
• technical meeting videos
• evidence grounding
• LLM-based QA