Improving Large Vision and Language Models by Learning from a Panel of Peers

2025-09-01

AI Summary

Current vision-language alignment methods either rely on costly human preference annotations or suffer from low-quality machine-generated data and hallucination-prone self-supervised approaches. To address these limitations, we propose PeerCL, a peer-collaborative learning framework that emulates human classroom-style collaborative learning without requiring large-scale manual annotation. PeerCL constructs a multi-model review panel that jointly generates responses, cross-evaluates outputs, and iteratively refines them via feedback-driven refinement. By integrating prompt engineering with a scalable inter-model evaluation mechanism, PeerCL effectively mitigates the challenges of high annotation cost, poor synthetic data quality, and hallucination. Evaluated across 15 benchmarks, PeerCL achieves an average score improvement from 48% to 57%, substantially outperforming existing self-supervised alignment methods. This demonstrates both the efficacy and scalability of the peer-evaluation paradigm for vision-language alignment.

Technology Category

• Computer Vision: Language and Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Traditional alignment methods for Large Vision and Language Models (LVLMs) primarily rely on human-curated preference data. Human-generated preference data is costly; machine-generated preference data is limited in quality; and self-supervised preference data often introduces hallucinations. To overcome these limitations, we propose a novel Panel-of-Peers learning framework inspired by collaborative learning among humans. This approach leverages a panel of LVLMs, each evaluating and learning from their collective outputs through an iterative self-improvement process. By simulating a peer review system, our models generate, assess, and refine outputs in response to a curated set of prompts, mimicking a classroom learning environment. We demonstrate that this methodology enhances model performance without requiring extensive human-labeled datasets. Our experiments show significant improvement across multiple benchmarks, demonstrating the potential of peer evaluations as a scalable alternative to self-supervised alignment. Notably, we show that Panel-of-Peers increases the average score on fifteen benchmarks from 48% to 57%

Problem

• Overcoming limitations of costly human-curated preference data
• Addressing quality issues with machine-generated preference data
• Reducing hallucinations in self-supervised alignment methods

Innovation

• Panel-of-Peers framework for collaborative LVLM learning
• Iterative self-improvement through simulated peer review system
• Scalable alternative to human-curated preference data