More than One Step at a Time: Designing Procedural Feedback for Non-visual Makeup Routines

2025-07-05

AI Summary

Visually impaired individuals face significant challenges in performing makeup independently—particularly in coordinating sequential steps, locating cosmetic products, and evaluating aesthetic outcomes—without visual feedback. Method: Through contextual interviews, video-based behavioral analysis, expert review, and qualitative coding, this study systematically elicited user needs and integrated professional makeup knowledge. Contribution/Results: We propose the first feedback requirement taxonomy specifically for non-visual makeup assistance, emphasizing embodied perception and haptics, and advocating hands-free, conversational, and context-aware procedural feedback. The study identifies three core technical challenges: smudging control, facial symmetry maintenance, and real-time aesthetic assessment. From these, we derive design principles supporting expressive and autonomous cosmetic practice. Our work establishes a theoretical foundation and actionable design guidelines for inclusive, accessibility-oriented beauty assistance systems.

Technology Category

• Humans and AI: AI for Accessibility
• Computer Vision: Bias, Fairness & Privacy
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Makeup plays a vital role in self-expression, identity, and confidence - yet remains an underexplored domain for assistive technology, especially for people with vision impairments. While existing tools support isolated tasks such as color identification or product labeling, they rarely address the procedural complexity of makeup routines: coordinating step sequences, managing product placement, and assessing the final look with accessible feedback. To understand the real-world process, we conducted a contextual inquiry with 15 visually impaired makeup users, capturing real-time makeup application behaviors and their step-by-step information needs and assessment approaches. Our findings reveal embodied, tactile-first strategies; persistent challenges in blending, symmetry, and assessment; and a desire for honest, real-time, goal-aligned feedback. We also interviewed five professional makeup artists, who reviewed participant makeup videos and provided expert responses to participant-raised questions and assessment practices. We contribute a taxonomy of feedback needs in non-visual makeup, and outline design implications for future assistive systems - emphasizing hands-free, conversational interaction and context-aware, procedural support for expressive and independent beauty practices.

Problem

• Addressing procedural complexity in non-visual makeup routines
• Providing real-time feedback for blending and symmetry challenges
• Designing hands-free assistive systems for independent makeup application

Innovation

• Hands-free conversational interaction for makeup
• Context-aware procedural support system
• Real-time goal-aligned feedback mechanism