Flow to Learn: Flow Matching on Neural Network Parameters

2025-03-25

AI Summary

Vision models lack context-driven conceptual learning capabilities analogous to those of large language models. Method: This paper introduces FLoWN, the first framework to incorporate flow matching into neural network parameter generation. FLoWN models conditional parameter flows in latent space, enabling dynamic synthesis of task-specific weights during inference. Contribution/Results: The approach achieves context-aware zero-shot and few-shot parameter generation, while preserving differentiability and cross-distribution generalization. Experiments demonstrate that FLoWN matches or surpasses baseline methods on in-distribution tasks; moreover, it significantly outperforms state-of-the-art approaches on out-of-distribution few-shot classification—delivering superior classifier initialization that directly addresses core meta-learning requirements.

Technology Category

• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Foundational language models show a remarkable ability to learn new concepts during inference via context data. However, similar work for images lag behind. To address this challenge, we introduce FLoWN, a flow matching model that learns to generate neural network parameters for different tasks. Our approach models the flow on latent space, while conditioning the process on context data. Experiments verify that FLoWN attains various desiderata for a meta-learning model. In addition, it matches or exceeds baselines on in-distribution tasks, provides better initializations for classifier training, and is performant on out-of-distribution few-shot tasks while having a fine-tuning mechanism to improve performance.

Problem

• Generating neural network parameters for diverse tasks
• Improving few-shot learning performance for images
• Enhancing meta-learning via flow matching on latent space

Innovation

• Flow matching model for neural parameters
• Latent space flow conditioning on context
• Fine-tuning mechanism for few-shot tasks