Memory Reviver: Supporting Photo-Collection Reminiscence for People with Visual Impairment via a Proactive Chatbot

2025-01-26

AI Summary

Visually impaired individuals struggle to independently engage in photo-based reminiscence due to limitations in existing technologies—particularly in semantic information organization and proactive conversational guidance. To address this, we propose an active chatbot framework tailored for visually impaired users, integrating a memory tree (a hierarchical semantic structure for organizing photo content) with a timing-aware proactive prompting mechanism, enabling natural language–driven, personalized reminiscence interactions. Technically, the framework unifies multimodal image understanding, hierarchical knowledge graph construction, dialogue state tracking, and dialogue-turn–aware dynamic content delivery. Evaluated with 12 visually impaired participants, our approach significantly improves reminiscence engagement, depth of photo understanding, and dialogue naturalness (p < 0.01). This work represents the first integration of structured memory modeling and proactive guidance strategies within photo-based reminiscence, advancing accessible human–AI interaction for memory support.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Computer Vision: Image and Video Retrieval
• Humans and AI: AI for Accessibility

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Reminiscing with photo collections offers significant psychological benefits but poses challenges for people with visual impairment (PVI). Their current reliance on sighted help restricts the flexibility of this activity. In response, we explored using a chatbot in a preliminary study. We identified two primary challenges that hinder effective reminiscence with a chatbot: the scattering of information and a lack of proactive guidance. To address these limitations, we present Memory Reviver, a proactive chatbot that helps PVI reminisce with a photo collection through natural language communication. Memory Reviver incorporates two novel features: (1) a Memory Tree, which uses a hierarchical structure to organize the information in a photo collection; and (2) a Proactive Strategy, which actively delivers information to users at proper conversation rounds. Evaluation with twelve PVI demonstrated that Memory Reviver effectively facilitated engaging reminiscence, enhanced understanding of photo collections, and delivered natural conversational experiences. Based on our findings, we distill implications for supporting photo reminiscence and designing chatbots for PVI.

Problem

• Visual Impairment
• Photo Recall Assistance
• Technological Insufficiency

Innovation

• Memory Reviver
• Memory Tree
• Proactive Strategy