Leveraging Metamemory Mechanisms for Enhanced Data-Free Code Generation in LLMs

2025-01-14

AI Summary

To address the low reliability of large language models (LLMs) in code generation under zero-annotation settings, this paper proposes the M²WF framework—the first to incorporate human-like metamemory mechanisms into code generation, enabling self-driven evaluation and iterative optimization without ground-truth reference solutions. M²WF establishes a closed “generate–evaluate–exploit” loop: an LLM autonomously produces candidate code, validates it via program execution feedback, performs confidence-weighted sampling, and dynamically reconstructs prompts—eliminating reliance on manual annotations or static exemplar sets. Evaluated on benchmarks including HumanEval and StudentEval, M²WF achieves a 12.3% absolute improvement in Pass@1 over state-of-the-art zero-shot and few-shot methods, demonstrates strong cross-task generalization, and operates entirely without training data.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Automated code generation using large language models (LLMs) has gained attention due to its efficiency and adaptability. However, real-world coding tasks or benchmarks like HumanEval and StudentEval often lack dedicated training datasets, challenging existing few-shot prompting approaches that rely on reference examples. Inspired by human metamemory-a cognitive process involving recall and evaluation-we present a novel framework (namely M^2WF) for improving LLMs' one-time code generation. This approach enables LLMs to autonomously generate, evaluate, and utilize synthetic examples to enhance reliability and performance. Unlike prior methods, it minimizes dependency on curated data and adapts flexibly to various coding scenarios. Our experiments demonstrate significant improvements in coding benchmarks, offering a scalable and robust solution for data-free environments. The code and framework will be publicly available on GitHub and HuggingFace.

Problem

• Large Language Models
• Code Generation
• Data Scarcity

Innovation

• M^2WF Method
• Code Generation
• Autonomous Evaluation