No Test Cases, No Problem: Distillation-Driven Code Generation for Scientific Workflows

2026-04-24

AI Summary

This work addresses the challenge of generating scientific workflow code in the absence of input/output test cases, which hinders optimization via execution feedback. To this end, the authors propose MOSAIC, a novel framework that achieves effective code generation without I/O supervision for the first time. MOSAIC leverages a training-free multi-agent collaboration strategy, integrating structured problem decomposition, domain-specific exemplar guidance, and teacher-student knowledge distillation. It further introduces an innovative Consolidated Context Window mechanism to maintain consistency across multi-step reasoning and mitigate hallucinations. Evaluated on the SciCode benchmark, MOSAIC substantially improves the accuracy, executability, and numerical precision of generated code, outperforming all existing methods across the board.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Existing multi-agent Large Language Model (LLM) frameworks for code generation typically use execution feedback and improve iteratively using Input/Output (I/O) test cases. However, this does not work for scientific workflows, where I/O test cases do not exist, and generating them requires solving the very problem at hand. To address this, we introduce MOSAIC, a training-free multi-agent framework for scientific code generation without I/O supervision. Instead of execution feedback, MOSAIC employs a student-teacher knowledge distillation framework that grounds generation through domain-specific examples and structured problem decomposition. To further mitigate hallucinations across chained subproblems, we introduce a Consolidated Context Window (CCW) for maintaining consistent reasoning across agents. Experiments on the SciCode benchmark show that MOSAIC improves accuracy, executability, and numerical precision over existing approaches while relying on lightweight models.

Problem

• scientific code generation
• I/O test cases
• multi-agent LLM
• knowledge distillation
• hallucination mitigation

Innovation

• knowledge distillation
• multi-agent LLM
• scientific code generation
• Consolidated Context Window
• test-case-free