Which Algorithm Specification Formats Help Language Models Implement Machine Learning Algorithms?

2026-07-03

AI Summary

This study investigates how the format of algorithmic descriptions influences the accuracy of machine learning algorithm implementations generated by large language models (LLMs), with a focus on critical yet often implicit details such as interfaces, computational steps, numerical rules, and boundary behaviors. Through controlled experiments across multiple models (GPT-4o mini, Gemma 2 27B, Llama 3.2 3B), five ML algorithms, and seven description formats—including LaTeX pseudocode, YAML, and Python code stubs—the authors evaluate implementation correctness using fine-grained hidden tests. Results reveal that content clarity outweighs format per se: under core information conditions, LaTeX pseudocode yields the best performance, followed by YAML and plain text; with complete information, some models become format-agnostic, and code stubs offer no significant advantage. This work is the first to systematically demonstrate the pivotal role of algorithmic description readability in LLM-based implementation and offers writing guidelines optimized for AI interpretability.

Abstract

Large language models (LLMs) are increasingly used to implement algorithms from research manuscripts, but papers often leave implementation choices implicit. This study examines how the written format of an algorithm specification affects first-pass LLM implementation accuracy. We compare ordinary prose, LaTeX algorithm-style pseudocode, PDF-like extracted pseudocode, Markdown fields, YAML-like specifications, JSON-like specifications, and Python code stubs across five machine learning tasks, three models, and four experimental settings, yielding 4,020 generated implementations. Hidden tests evaluate details that often determine correctness, including tie-breaking, array shapes, numerical rules, return structures, and invalid-input behavior. Under the core-information setting, LaTeX algorithm-style pseudocode has the largest average format effect, with YAML-like specifications and ordinary prose close behind. Under complete information, GPT-5.4 mini shows no format differences in the matched comparisons, whereas Gemma 3 4B and Llama 3.2 3B still do. Code stubs do not consistently improve correctness despite specifying the function signature. The results support a writing recommendation: authors should state the interface, computation steps, numerical rules, and boundary-case behavior explicitly, instead of relying on a particular surface format to carry those details.

Problem

• algorithm specification
• large language models
• implementation accuracy
• machine learning algorithms
• pseudocode formats

Innovation

• algorithm specification formats
• large language models
• implementation accuracy
• hidden tests
• machine learning reproducibility