Reciprocal Co-Training (RCT): Coupling Gradient-Based and Non-Differentiable Models via Reinforcement Learning

2026-03-24

arXiv.org

AI Summary

This work addresses the challenge of integrating language models with non-differentiable models such as random forests, which has been hindered by fundamental differences in representation and training paradigms. The authors propose the first general framework enabling bidirectional collaborative training between gradient-based and non-differentiable models. Specifically, tabular data are textualized and fed into a language model, whose embeddings enrich the feature space of the random forest. In turn, the calibrated outputs of the random forest serve as reward signals to guide policy updates of the language model via reinforcement learning. By synergistically combining textualization, embedding augmentation, hybrid reward design, and dimensionality control, the approach significantly enhances the performance of ClinicalBERT and Qwen2-7B-Instruct across three medical datasets. SHAP analysis further confirms the critical contribution of the language model’s learned representations to predictive accuracy.

Abstract

Large language models (LLMs) and classical machine learning methods offer complementary strengths for predictive modeling, yet their fundamentally different representations and training paradigms hinder effective integration: LLMs rely on gradient-based optimization over textual data, whereas models such as Random Forests (RF) employ non-differentiable feature partitioning. This work introduces a reciprocal co-training framework that couples an LLM with an RF classifier via reinforcement learning, creating an iterative feedback loop in which each model improves using signals from the other. Tabular data are reformulated into standardized textual representations for the LLM, whose embeddings augment the RF feature space, while calibrated RF probability estimates provide feedback signals that guide reinforcement learning updates of the LLM. Experiments across three medical datasets demonstrate consistent performance gains for both models, with particularly strong effects for the LLM. Ablation analyses show that iterative refinement, hybrid reward design, and dimensionality control jointly contribute to these gains. The proposed framework provides a general mechanism that allows incompatible model families to leverage each other's strengths through bidirectional adaptation.

Problem

• Reciprocal Co-Training
• Language Models
• Non-Differentiable Models
• Model Integration
• Reinforcement Learning

Innovation

• Reciprocal Co-Training
• Reinforcement Learning
• Language Models
• Random Forest
• Tabular-to-Text Representation