Position: Assistive Agents Need Accessibility Alignment

2026-05-13

AI Summary

This study addresses the frequent failure of existing agent systems in visually impaired contexts, where assumptions of sighted users lead to high verification costs, low fault tolerance, and restrictive interactions. For the first time, accessibility needs are integrated into the core objective of agent alignment, proposing a full lifecycle framework encompassing user research, design, deployment, and iteration. Drawing on an analysis of 778 visual assistance tasks and synthesizing human-centered design with AI alignment methodologies, the work reconfigures the agent development pipeline. It exposes systemic shortcomings of current systems in assistive scenarios and establishes visual impairment tasks as critical stress tests for agent robustness and inclusivity, thereby advancing a new paradigm of inclusive AI grounded in the realities of diverse end users.

Abstract

Assistive agents for Blind and Visually Impaired (BVI) users require accessibility alignment as a first-class design objective. Despite rapid progress in agentic AI, most systems are designed and evaluated under assumptions of sighted interaction, low-cost verification, and tolerable trial-and-error, leading to systematic failures in assistive scenarios that cannot be resolved by model scaling or post-hoc interface adaptations alone. Drawing on an analysis of 778 assistance task instances from prior work, we show that current agentic AI remain prone to failure in assistive scenarios due to mismatches between sighted-user design assumptions and the verification, risk, and interaction constraints faced by BVI users. We argue that accessibility should be treated as an alignment problem rather than a peripheral usability concern. To this end, we introduce accessibility alignment and propose a lifecycle-oriented design pipeline for accessibility-aligned assistive agents, spanning user research, system design, deployment and post-deployment iteration. We conclude that BVI-centered assistive tasks provide a critical stress test for agentic AI and motivate a broader shift toward inclusive agent design.

Problem

• accessibility alignment
• assistive agents
• Blind and Visually Impaired
• agentic AI
• inclusive design

Innovation

• accessibility alignment
• assistive agents
• blind and visually impaired (BVI)
• inclusive agent design
• lifecycle-oriented design