TactileEval: A Step Towards Automated Fine-Grained Evaluation and Editing of Tactile Graphics

2026-04-20

AI Summary

This study addresses the limitation of existing tactile graphic datasets, which provide only coarse-grained quality ratings and lack actionable feedback for blind and visually impaired learners. To bridge this gap, the authors propose the first fine-grained quality assessment framework, comprising a three-stage pipeline: defining five categories of quality issues based on expert critiques, constructing a large-scale dataset via structured crowdsourced annotations, and developing an automated diagnostic and editing system that combines ViT-L/14 feature probing with GPT-Image-1 enhanced by domain-aligned prompting templates. Evaluated across 30 tasks, the method achieves an average accuracy of 85.70%, demonstrating the perceptual validity of the proposed assessment scheme and enabling category-specific automatic correction of tactile graphics, thereby advancing the automated generation of accessible educational resources.

Technology Category

• Humans and AI: AI for Accessibility
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

Tactile graphics require careful expert validation before reaching blind and visually impaired (BVI) learners, yet existing datasets provide only coarse holistic quality ratings that offer no actionable repair signal. We present TactileEval, a three-stage pipeline that takes a first step toward automating this process. Drawing on expert free-text comments from the TactileNet dataset, we establish a five-category quality taxonomy; encompassing view angle, part completeness, background clutter, texture separation, and line quality aligned with BANA standards. We subsequently gathered 14,095 structured annotations via Amazon Mechanical Turk, spanning 66 object classes organized into six distinct families. A reproducible ViT-L/14 feature probe trained on this data achieves 85.70% overall test accuracy across 30 different tasks, with consistent difficulty ordering suggesting the taxonomy suggesting the taxonomy captures meaningful perceptual structure. Building on these evaluations, we present a ViT-guided automated editing pipeline that routes classifier scores through family-specific prompt templates to produce targeted corrections via gpt-image-1 image editing. Code, data, and models are available at https://TactileEval.github.io/

Problem

• tactile graphics
• quality evaluation
• blind and visually impaired
• automated editing
• fine-grained annotation

Innovation

• tactile graphics
• fine-grained evaluation
• automated editing
• vision transformer
• accessibility