LLMs can construct powerful representations and streamline sample-efficient supervised learning

2026-03-12

AI Summary

This work addresses the limitations of handcrafted input representations in supervised learning on complex heterogeneous data—such as time series, text, and structured records—by proposing a large language model (LLM)-based agent pipeline. The approach automatically derives global normalization rules from a small set of diverse textual serialization examples and combines them with task-conditioned local rules to construct efficient, auditable, and low-cost tabular input representations. Evaluated on 15 clinical tasks in the EHRSHOT benchmark, the method significantly outperforms conventional count-based feature models, naive LLM text serialization baselines, and large-scale pretrained clinical foundation models, demonstrating strong effectiveness and generalization capability in few-shot medical settings.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

As real-world datasets become increasingly complex and heterogeneous, supervised learning is often bottlenecked by input representation design. Modeling multimodal data for downstream tasks, such as time-series, free text, and structured records, often requires non-trivial domain-specific engineering. We propose an agentic pipeline to streamline this process. First, an LLM analyzes a small but diverse subset of text-serialized input examples in-context to synthesize a global rubric, which acts as a programmatic specification for extracting and organizing evidence. This rubric is then used to transform naive text-serializations of inputs into a more standardized format for downstream models. We also describe local rubrics, which are task-conditioned summaries generated by an LLM. Across 15 clinical tasks from the EHRSHOT benchmark, our rubric-based approaches significantly outperform traditional count-feature models, naive text-serialization-based LLM baselines, and a clinical foundation model, which is pretrained on orders of magnitude more data. Beyond performance, rubrics offer several advantages for operational healthcare settings such as being easy to audit, cost-effectiveness to deploy at scale, and they can be converted to tabular representations that unlock a swath of machine learning techniques.

Problem

• supervised learning
• input representation
• multimodal data
• domain-specific engineering
• heterogeneous datasets

Innovation

• large language models
• rubric-based representation
• supervised learning
• multimodal data
• clinical NLP