Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline

2026-06-27

AI Summary

This study addresses the limitations of general-purpose large language models (LLMs) in specialized domains such as transportation engineering, where insufficient understanding of technical standards and domain-specific semantics hinders performance on complex tasks. To overcome this, the authors construct a domain-specific corpus comprising U.S. transportation engineering manuals, design guidelines, and regulatory documents, and apply continual pretraining to six state-of-the-art LLMs within a unified low-rank adaptation (LoRA) framework. This work introduces the first reproducible paradigm for developing domain-tailored generative AI agents. Experimental results demonstrate that Qwen2.5-7B and LLaMA-3.1-8B achieve superior domain alignment and response quality, confirming that LoRA-based fine-tuning with authoritative technical documentation significantly enhances model comprehension and reasoning capabilities in specialized engineering contexts.

Abstract

Recent advancements in generative artificial intelligence (AI) and large language models (LLMs) have shown significant promise in automating complex reasoning, summarization, and question-answering tasks. However, the effectiveness of general-purpose LLMs in specialized engineering domains remains limited due to insufficient exposure to technical standards, engineering terminology, and domain-specific semantics. This study proposes a systematic approach to developing a customized generative AI agent for transportation engineering applications. A curated corpus of U.S. transportation manuals, design guidelines, and regulatory documents is used to conduct continued pretraining of six state-of-the-art LLMs through a unified low-rank adaptation (LoRA) framework. The training process is monitored to ensure convergence and model stability. Performance is evaluated using standard natural language processing metrics, including BLEU-4 and ROUGE, with Qwen2.5-7B and LLaMA-3.1-8B demonstrating the highest domain alignment and response quality. Results validate the effectiveness of LoRA-based adaptation in improving LLM performance on technical content interpretation and context-specific reasoning. This work contributes a reproducible development framework for constructing domain-specialized generative AI agents, supporting broader deployment in transportation research, design, planning, and policy analysis.

Problem

• generative AI
• large language models
• domain adaptation
• transportation engineering
• technical semantics

Innovation

• customized generative AI
• transportation engineering
• continued pretraining
• low-rank adaptation (LoRA)
• domain-specific LLMs