An Experience Report on Regression-Free Repair of Deep Neural Network Model

2023-03-01

IEEE International Conference on Software Analysis, Evolution, and Reengineering

AI Summary

In high-reliability industrial systems—such as automotive image security applications—updating deep neural network (DNN) models often induces performance regression on critical classes, jeopardizing safety and reliability. Method: This paper proposes NeuRecover, the first category-specific, zero-regression neural network repair framework. It introduces a customized objective function and a class-level constrained optimization mechanism, validated empirically on real-world automotive image data. Contribution/Results: Applied to Fujitsu’s production automotive model update task, NeuRecover achieves strict zero regression for safety-critical classes—i.e., no performance degradation—while preserving primary task accuracy. The work systematically identifies practical challenges in zero-regression repair within industrial settings and establishes a viable, deployable technical paradigm for trustworthy AI model evolution.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Adversarial Attacks & Robustness
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Systems based on Deep Neural Networks (DNNs) are increasingly being used in industry. In the process of system operation, DNNs need to be updated in order to improve their performance. When updating DNNs, systems used in companies that require high reliability must have as few regressions as possible. Since the update of DNNs has a data-driven nature, it is difficult to suppress regressions as expected by developers. This paper identifies the requirements for DNN updating in industry and presents a case study using techniques to meet those requirements. In the case study, we worked on satisfying the requirement to update models trained on car images collected in Fujitsu assuming security applications without regression for a specific class. We were able to suppress regression by customizing the objective function based on NeuRecover, a DNN repair technique. Moreover, we discuss some of the challenges identified in the case study.

Problem

• Minimizing regressions during DNN updates in high-reliability systems.
• Customizing objective functions to suppress specific class regressions.
• Addressing challenges in regression-free DNN repair for security applications.

Innovation

• Customized objective function for DNN repair
• NeuRecover technique to suppress regression
• Case study on car image models update