Training-free Composition of Pre-trained GFlowNets for Multi-Objective Generation

2026-02-24

AI Summary

Existing approaches to multi-objective generation typically require retraining GFlowNets for each new set of objectives, resulting in high computational costs and limited flexibility. This work proposes a training-free, inference-time mixing strategy that combines pretrained GFlowNets to flexibly adapt to both linear and nonlinear multi-objective reward functions. It is the first method capable of exactly recovering the true target distribution under linear objectives without any fine-tuning, and it provides a theoretical analysis of the approximation error for the resulting distribution. Empirical evaluations on 2D grid navigation and molecular design tasks demonstrate that the proposed approach matches the performance of baseline methods that require additional training, while substantially reducing computational overhead.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial. Further extending GFlowNets to multi-objective settings has attracted growing interest since real-world applications often involve multiple, conflicting objectives. However, existing approaches require additional training for each set of objectives, limiting their applicability and incurring substantial computational overhead. We propose a training-free mixing policy that composes pre-trained GFlowNets at inference time, enabling rapid adaptation without finetuning or retraining. Importantly, our framework is flexible, capable of handling diverse reward combinations ranging from linear scalarization to complex non-linear logical operators, which are often handled separately in previous literature. We prove that our method exactly recovers the target distribution for linear scalarization and quantify the approximation quality for nonlinear operators through a distortion factor. Experiments on a synthetic 2D grid and real-world molecule-generation tasks demonstrate that our approach achieves performance comparable to baselines that require additional training.

Problem

• GFlowNets
• multi-objective generation
• training-free composition
• reward composition
• scientific discovery

Innovation

• Training-free Composition
• Multi-Objective GFlowNets
• Inference-time Mixing
• Nonlinear Reward Combination
• Generative Flow Networks