CoKnowledge: Supporting Assimilation of Time-synced Collective Knowledge in Online Science Videos

2025-02-06

AI Summary

Disorganized danmaku (real-time overlaid comments) in scientific videos impede effective absorption of collective knowledge. Method: We propose CoKnowledge—a novel end-to-end system that integrates temporal-aware danmaku filtering, semantic classification and clustering, dynamic keyframe-based video summarization, domain-adapted knowledge graph construction, and enhanced interactive danmaku visualization. It systematically addresses cognitive barriers to danmaku-based knowledge absorption through a tripartite paradigm: temporal synchronization, structural representation, and interactive support. Contribution/Results: A user study (N=24) demonstrates that CoKnowledge significantly improves collective knowledge comprehension (p<0.01) and recall accuracy (+37.2%) over raw danmaku, empirically validating the efficacy and innovation of structured, temporally grounded knowledge support for science communication.

Technology Category

• Data Mining & Knowledge Management: Data Visualization & Summarization
• Computer Vision: Video Understanding & Activity Analysis
• Natural Language Processing: Summarization

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Interdisciplinary science discovery with web data mining
• Security and Privacy: Data transparency and provenance

Abstract

Danmaku, a system of scene-aligned, time-synced, floating comments, can augment video content to create 'collective knowledge'. However, its chaotic nature often hinders viewers from effectively assimilating the collective knowledge, especially in knowledge-intensive science videos. With a formative study, we examined viewers' practices for processing collective knowledge and the specific barriers they encountered. Building on these insights, we designed a processing pipeline to filter, classify, and cluster danmaku, leading to the development of CoKnowledge - a tool incorporating a video abstract, knowledge graphs, and supplementary danmaku features to support viewers' assimilation of collective knowledge in science videos. A within-subject study (N=24) showed that CoKnowledge significantly enhanced participants' comprehension and recall of collective knowledge compared to a baseline with unprocessed live comments. Based on our analysis of user interaction patterns and feedback on design features, we presented design considerations for developing similar support tools.

Problem

• Enhance assimilation of collective knowledge in videos
• Filter and classify chaotic danmaku comments effectively
• Improve comprehension and recall in science video content

Innovation

• Time-synced danmaku processing
• Knowledge graphs integration
• Video abstract enhancement