Generalized Parallel Scaling with Interdependent Generations

2025-10-01

AI Summary

In parallel LLM inference, conventional N-way independent response generation leads to imbalanced resource utilization and prevents latent information sharing across sequences, thereby limiting response quality and consistency. To address this, we propose Bridge—a lightweight tensor-level mechanism that reconstructs batched hidden states to enable implicit cross-sequence information sharing, thereby introducing interdependence among parallel responses. Bridge adds only 2.8%–5.1% parameters and is the first method to support arbitrary-width, scalable dependency-aware parallel inference while remaining compatible with downstream aggregation. Integrated with reinforcement learning and verifiable reward training, a single model trained with Bridge generalizes seamlessly across varying parallelism degrees. On reinforcement learning benchmarks, it achieves up to 50% relative improvement in average accuracy and significantly enhances consistency among correct responses.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Parallel LLM inference scaling involves sampling a set of $N>1$ responses for a single input prompt. However, these $N$ parallel responses tend to be generated independently from each other, partitioning compute resources and leaving potentially useful information in one generation untapped by others. This is in contrast to response length scaling where past computation is used in all future steps. For higher quality responses and response sets, we propose Bridge to generate interdependent responses in parallel by rethinking batched LLM hidden states as holistic tensors rather than independent slices. With only a small amount (2.8%-5.1%) of new parameters, Bridge improves the relative mean accuracy gains from reinforcement learning with verifiable rewards by up to 50% and boosts consistency of correct responses. Trained once, Bridge scales to any generation width, all with greater performance than independent generations, unlocking a more general mode of parallel scaling that effectively leverages information between sequences, compatible with any post-generation aggregation technique.

Problem

• Enabling interdependent parallel LLM responses through holistic tensor processing
• Overcoming independent generation limitations by sharing computational information
• Improving response quality and consistency with minimal additional parameters

Innovation

• Generates interdependent parallel responses via holistic tensors
• Adds minimal new parameters to enable cross-sequence information sharing
• Improves accuracy and consistency while maintaining generation width flexibility