Making AI Drafts Count: A Quality Threshold in Audio Description Workflows

2026-05-06

AI Summary

This study investigates how the quality of AI-generated drafts affects the efficiency and effectiveness of human editing in audio description tasks, proposing a “quality threshold” principle: AI drafts significantly enhance human-AI collaboration only when their quality meets a minimum level commensurate with the visual complexity of the source content. To evaluate this, we developed the GenAD generation pipeline and the RefineAD editing interface, conducting controlled experiments to quantify differences between creating descriptions from scratch and editing AI drafts of varying quality in terms of task duration, cognitive load, and output quality. Results demonstrate that high-quality drafts reduce task time by over 50% and substantially lower cognitive load, whereas low-quality drafts yield limited benefits, thereby confirming the existence and critical role of the quality threshold, particularly for complex visual content.

Abstract

Audio description (AD) narrates visual elements in video for blind and low-vision audiences. Recent work has shown that giving novice describers an AI-generated draft to start from helps produce higher-quality AD and lowers the barrier to entry. What remains an open question is how draft quality shapes the editing process. We investigate this through GenAD, an AD generation pipeline that incorporates accessibility guidelines and contextual video information, and RefineAD, an editing interface for human revisions. Human-AI contributions are measured across text, timing, and delivery. In a within-subjects study, we compared authoring from scratch against editing AI drafts of varying quality. GenAD drafts cut completion time by more than half and significantly reduced cognitive load. In contrast, baseline drafts generated from simple, unguided prompts offered only modest benefits, pointing to a minimum quality threshold for effectiveness. Qualitative findings suggest this threshold is content-dependent; as visual complexity increases, so does the quality needed from AI drafts. We propose this as a design principle: effective AI assistance should clear a quality threshold suited to the target content, rather than simply be present.

Problem

• audio description
• AI draft quality
• human-AI collaboration
• quality threshold
• accessibility

Innovation

• quality threshold
• audio description
• human-AI collaboration
• GenAD
• accessibility