Reward Lightning: Fast Video Generation via Homologous Preference Distillation

2026-07-04

AI Summary

This work addresses the conflict between preference alignment and distillation-based acceleration in video diffusion models, which often arises from misaligned optimization objectives. To resolve this, the authors propose Reward Lightning, a unified framework that enables synergistic optimization by sharing a common latent representation space. The approach introduces two key innovations: Homologous Preference Distillation (HPD) and a Latent Reward Model (LRM), which jointly mitigate gradient conflicts. Experimental results demonstrate that LRM outperforms baseline methods by 11.0%–14.7% in preference accuracy. Moreover, the framework achieves high-fidelity video generation in just 1–4 denoising steps, yielding a 2.1% improvement in average VBench score and establishing significant gains in text alignment, motion dynamics, and visual quality.

Abstract

Achieving simultaneous preference alignment and distillation acceleration in video diffusion models remains an open challenge. Existing methods optimize the two objectives over mismatched representation spaces, where improving one objective often compromises the other. To overcome this, we propose Reward Lightning, a unified framework that aligns and accelerates a video diffusion model within a single shared representation. Its central principle is homology: both objectives are evaluated on identical latent features, which mitigates the gradient conflicts that arise when they are optimized over disjoint representations. As a foundational component, we first introduce a latent reward model (LRM) that scores videos directly in the latent space, without decoding back to the pixel space. Building on the LRM, homologous preference distillation (HPD) reuses this shared backbone to perform adversarial distillation and preference alignment jointly, yielding few-step generators that remain faithful and well aligned. Extensive experiments demonstrate that the LRM surpasses pixel-level and latent-level reward baselines by $11.0\%$ and $14.7\%$ in preference accuracy, and that Reward Lightning generates high-fidelity videos in merely $1$ to $4$ steps, improving the average VBench score by $2.1\%$ while leading in text alignment, motion quality, and visual quality. Project page: https://reward-lightning.github.io.

Problem

• preference alignment
• distillation acceleration
• video diffusion models
• representation space
• gradient conflict

Innovation

• homologous preference distillation
• latent reward model
• video diffusion
• preference alignment
• distillation acceleration