Adapting, Fast and Slow: Transportable Circuits for Few-Shot Learning

2025-12-27

AI Summary

Cross-domain few-shot generalization suffers from insufficient structural constraints between source and target domains and extreme label scarcity in the target domain. Method: This paper proposes a modular local prediction circuit framework grounded in causal transferability theory. After training on the source domain, it enables zero-shot adaptation to unseen target domains without requiring any target-domain labels. The framework innovatively integrates causal graph modeling with a mechanism-sharing discriminator to achieve, for the first time, zero-shot compositional generalization. Contribution/Results: We introduce the circuit transferability criterion—a theoretical sufficient condition for few-shot learnability—and establish a quantitative relationship between circuit complexity and transfer efficiency. Empirical results show that smaller circuits achieve substantial generalization gains with only 1–5 target-domain samples, offering an interpretable and verifiable paradigm for cross-domain few-shot learning.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Generalization across the domains is not possible without asserting a structure that constrains the unseen target domain w.r.t. the source domain. Building on causal transportability theory, we design an algorithm for zero-shot compositional generalization which relies on access to qualitative domain knowledge in form of a causal graph for intra-domain structure and discrepancies oracle for inter-domain mechanism sharing. extit{Circuit-TR} learns a collection of modules (i.e., local predictors) from the source data, and transport/compose them to obtain a circuit for prediction in the target domain if the causal structure licenses. Furthermore, circuit transportability enables us to design a supervised domain adaptation scheme that operates without access to an explicit causal structure, and instead uses limited target data. Our theoretical results characterize classes of few-shot learnable tasks in terms of graphical circuit transportability criteria, and connects few-shot generalizability with the established notion of circuit size complexity; controlled simulations corroborate our theoretical results.

Problem

• Zero-shot compositional generalization using causal transportability
• Supervised domain adaptation without explicit causal structure
• Characterizing few-shot learnable tasks via graphical circuit transportability

Innovation

• Uses causal transportability theory for zero-shot generalization
• Learns modular predictors and composes them into circuits
• Enables domain adaptation without explicit causal structure