Majority Voting for Code Generation

2026-04-16

AI Summary

This work addresses the limited accuracy and robustness of large language models in code generation under unlabeled test-time scenarios by proposing Functional Majority Voting (FMV). FMV constructs a consensus of functional equivalence among multiple generated programs based on their runtime execution signatures over test inputs, selecting a representative solution. Notably, it is the first method to employ such a consensus mechanism as an aggregation strategy for test-time reinforcement learning. Requiring no additional training or substantial computational overhead, FMV significantly improves pass@1 performance on LiveCodeBench, demonstrating its effectiveness as an efficient test-time inference mechanism. While it does not surpass the inherent capabilities of the base model, FMV establishes a novel paradigm for unsupervised code generation.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We investigate Functional Majority Voting (FMV), a method based on functional consensus for code generation with Large Language Models, which identifies a representative solution from multiple generations using their runtime execution signatures on test inputs. We find that FMV is an effective test-time inference strategy, substantially boosting performance on LiveCodeBench without a large compute overhead. Furthermore, we extend the utility of functional consensus and apply it as an aggregation strategy for label-free Test-Time Reinforcement Learning. We demonstrate that this increases pass@1 on holdout tasks, but find no evidence of self-improvement beyond the base model's performance ceiling.

Problem

• Code Generation
• Functional Consensus
• Majority Voting
• Test-Time Inference
• Reinforcement Learning

Innovation

• Functional Majority Voting
• code generation
• test-time inference
• functional consensus
• Test-Time Reinforcement Learning