N-Version Programming with Coding Agents

2026-06-18

AI Summary

This study investigates whether N-version programming can mitigate common-mode failures in AI-based code generation through diversity in models, systems, and programming languages. Building upon the Knight–Leveson experimental framework, the research employs 48 AI agents to produce multilingual implementations and evaluates their fault tolerance under one million randomly generated test inputs. The first systematic validation of this approach reveals that specification ambiguity is the primary source of common-mode failures. Employing a three-version majority voting scheme significantly reduces the average number of failures from 387.44 to 130.99, with 11,844 N-version units achieving zero failures. These findings demonstrate the substantial engineering value of N-version programming in enhancing the reliability of AI-generated code.

Abstract

This paper revisits the classical concept on N-version programming in the setting of contemporary AI coding agents. Revisiting the seminal Knight-Leveson experiment, we study whether diversity across agent systems, models, and implementation languages creates diverse failure modes. Using the Knight-Leveson's, Launch Interceptor Program Specification, we evaluate 48 agent-generated implementations on a shared oracle and a campaign of 1,000,000 randomized test inputs. The results show substantial common-mode failure, along the findings of Knight-Leveson. Further analysis that many of those co-occuring failures can be traced to where is specification is particularly hard or ambiguous. We also demonstrate that diversity from coding agents provides practical benefit: across majority voting three-version units, the mean failure count drops from 387.44 for single versions to 130.99 for triples, and 11,844 N-version units exhibit zero observed failures. Our original results is the strongest evidence to date that N-Version Programming with coding agents is a useful engineering strategy.

Problem

• N-Version Programming
• AI coding agents
• common-mode failure
• software diversity
• fault tolerance

Innovation

• N-Version Programming
• AI coding agents
• common-mode failure
• majority voting
• software diversity