Vision-Language Foundation Models for Comprehensive Automated Pavement Condition Assessment

2026-04-09

AI Summary

This work addresses the limitations of existing general-purpose vision-language models in specialized domains such as pavement condition assessment, which demand precise technical terminology, structured reasoning, and strict adherence to engineering standards. To bridge this gap, the authors propose a novel instruction-driven AI paradigm tailored for infrastructure applications. They introduce PaveInstruct, a comprehensive dataset encompassing 32 distinct tasks, and perform large-scale instruction tuning by integrating nine heterogeneous pavement datasets to train PaveGPT—a domain-specific vision-language foundation model. Rigorously aligned with ASTM D6433 standards, PaveGPT demonstrates performance improvements exceeding 20% on spatial localization, reasoning, and generation tasks. The model offers transportation agencies a unified conversational assessment tool capable of replacing multiple specialized systems.

Technology Category

• Computer Vision: Large Vision Models
• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

General-purpose vision-language models demonstrate strong performance in everyday domains but struggle with specialized technical fields requiring precise terminology, structured reasoning, and adherence to engineering standards. This work addresses whether domain-specific instruction tuning can enable comprehensive pavement condition assessment through vision-language models. PaveInstruct, a dataset containing 278,889 image-instruction-response pairs spanning 32 task types, was created by unifying annotations from nine heterogeneous pavement datasets. PaveGPT, a pavement foundation model trained on this dataset, was evaluated against state-of-the-art vision-language models across perception, understanding, and reasoning tasks. Instruction tuning transformed model capabilities, achieving improvements exceeding 20% in spatial grounding, reasoning, and generation tasks while producing ASTM D6433-compliant outputs. These results enable transportation agencies to deploy unified conversational assessment tools that replace multiple specialized systems, simplifying workflows and reducing technical expertise requirements. The approach establishes a pathway for developing instruction-driven AI systems across infrastructure domains including bridge inspection, railway maintenance, and building condition assessment.

Problem

• vision-language models
• pavement condition assessment
• domain-specific instruction tuning
• engineering standards
• automated infrastructure inspection

Innovation

• instruction tuning
• vision-language foundation model
• pavement condition assessment
• domain-specific dataset
• ASTM-compliant AI