ASDA: Automated Skill Distillation and Adaptation for Financial Reasoning

2026-03-17

AI Summary

This work addresses the limitations of existing training-free methods in complex financial multi-step reasoning tasks and the high cost of fine-tuning large language models. The authors propose a novel training-free framework that preserves model weights while leveraging a teacher model to analyze student model failures, cluster error patterns, and automatically generate structured skill artifacts—comprising reasoning workflows, code templates, and examples—which are dynamically injected during inference to enhance performance. This approach is the first to produce financial reasoning skills that are human-readable, version-controllable, and compliant with Agent Skills standards, thereby enabling auditability and reuse. Evaluated on the FAMMA benchmark, the method improves accuracy by 17.33% on arithmetic reasoning and 5.95% on non-arithmetic reasoning, substantially outperforming current training-free alternatives.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Adapting large language models (LLMs) to specialized financial reasoning typically requires expensive fine-tuning that produces model-locked expertise. Training-free alternatives have emerged, yet our experiments show that leading methods (GEPA and ACE) achieve only marginal gains on the FAMMA financial reasoning benchmark, exposing the limits of unstructured text optimization for complex, multi-step domain reasoning. We introduce Automated Skill Distillation and Adaptation (ASDA), a framework that automatically generates structured skill artifacts through iterative error-corrective learning without modifying model weights. A teacher model analyzes a student model's failures on financial reasoning tasks, clusters errors by subfield and error type, and synthesizes skill files containing reasoning procedures, code templates, and worked examples, which are dynamically injected during inference. Evaluated on FAMMA, ASDA achieves up to +17.33% improvement on arithmetic reasoning and +5.95% on non-arithmetic reasoning, substantially outperforming all training-free baselines. The resulting skill artifacts are human-readable, version-controlled, and compatible with the Agent Skills open standard, offering any organization with a labeled domain dataset a practical and auditable path to domain adaptation without weight access or retraining.

Problem

• financial reasoning
• domain adaptation
• training-free adaptation
• large language models
• structured skill artifacts

Innovation

• Automated Skill Distillation
• Training-free Adaptation
• Structured Skill Artifacts
• Financial Reasoning
• Error-corrective Learning