Semantic Interaction for Narrative Map Sensemaking: An Insight-based Evaluation

2026-03-31

AI Summary

This study addresses the lack of empirical evaluation regarding the effectiveness of semantic interaction in narrative map comprehension. Through a user study, it compares the performance of timelines, basic narrative maps, and narrative maps enhanced with semantic interaction on narrative understanding tasks, investigating how analysts leverage interaction to refine AI-extracted narrative structures. The work provides the first empirical evidence that semantic interaction enhances narrative insight and identifies two distinct interaction strategies—corrective and augmentative—highlighting their potential as lightweight alternatives to model fine-tuning. Results show that the semantic interaction condition significantly outperforms the timeline baseline; although it does not differ statistically from the basic narrative map, it exhibits a large effect size (d > 0.8) and achieves comparable exploratory breadth with substantially fewer parameter adjustments.

Abstract

Semantic interaction (SI) enables analysts to incorporate their cognitive processes into AI models through direct manipulation of visualizations. While SI frameworks for narrative extraction have been proposed, empirical evaluations of their effectiveness remain limited. This paper presents a user study that evaluates SI for narrative map sensemaking, involving 33 participants under three conditions: a timeline baseline, a basic narrative map, and an interactive narrative map with SI capabilities. The results show that the map-based prototypes yielded more insights than the timeline baseline, with the SI-enabled condition reaching statistical significance and the basic map condition trending in the same direction. The SI-enabled condition showed the highest mean performance; differences between the map conditions were not statistically significant but showed large effect sizes (d > 0.8), suggesting that the study was underpowered to detect them. Qualitative analysis identified two distinct SI approaches-corrective and additive-that enable analysts to impose quality judgments and organizational structure on extracted narratives. We also find that SI users achieved comparable exploration breadth with less parameter manipulation, suggesting that SI serves as an alternative pathway for model refinement. This work provides empirical evidence that map-based representations outperform timelines for narrative sensemaking, along with qualitative insights into how analysts use SI for narrative refinement.

Problem

• Semantic Interaction
• Narrative Sensemaking
• Narrative Map
• User Study
• Insight Evaluation

Innovation

• Semantic Interaction
• Narrative Map
• Sensemaking
• User Study
• Model Refinement