Retrieval-Augmented Fine-Tuning With Preference Optimization For Visual Program Generation

2025-02-23

AI Summary

Low generation accuracy of ladder diagram (LD) visual programming language in industrial automation, compounded by the inadequacy of existing prompting methods for complex domain-specific configurations. Method: We propose a two-stage training paradigm: (1) retrieval-augmented fine-tuning (RAFT) to enhance generalization by leveraging high-frequency subroutine patterns in industrial LDs; and (2) graph-editing–based direct preference optimization (DPO), which automatically constructs semantically consistent preference pairs to mitigate annotation scarcity and configuration complexity. Contribution/Results: This work is the first to synergistically integrate RAFT and DPO for visual program generation and introduces the first graph-editing–driven preference construction method tailored to LDs. On a real-world LD dataset, our approach achieves over 10% higher program-level accuracy than supervised fine-tuning, and even small models outperform large-model prompting methods—significantly improving reliability and practicality of industrial code generation.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Visual Reasoning & Symbolic Representations
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Visual programming languages (VPLs) allow users to create programs through graphical interfaces, which results in easier accessibility and their widespread usage in various domains. To further enhance this accessibility, recent research has focused on generating VPL code from user instructions using large language models (LLMs). Specifically, by employing prompting-based methods, these studies have shown promising results. Nevertheless, such approaches can be less effective for industrial VPLs such as Ladder Diagram (LD). LD is a pivotal language used in industrial automation processes and involves extensive domain-specific configurations, which are difficult to capture in a single prompt. In this work, we demonstrate that training-based methods outperform prompting-based methods for LD generation accuracy, even with smaller backbone models. Building on these findings, we propose a two-stage training strategy to further enhance VPL generation. First, we employ retrieval-augmented fine-tuning to leverage the repetitive use of subroutines commonly seen in industrial VPLs. Second, we apply direct preference optimization (DPO) to further guide the model toward accurate outputs, using systematically generated preference pairs through graph editing operations. Extensive experiments on real-world LD data demonstrate that our approach improves program-level accuracy by over 10% compared to supervised fine-tuning, which highlights its potential to advance industrial automation.

Problem

• Enhance Ladder Diagram generation accuracy.
• Propose retrieval-augmented fine-tuning for VPLs.
• Apply direct preference optimization for model guidance.

Innovation

• Retrieval-augmented fine-tuning for subroutines
• Direct preference optimization for accuracy
• Graph editing for preference pair generation