Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment

2026-07-15

AI Summary

This work addresses the challenges of video aesthetic assessment, which are hindered by subjectivity and the absence of large-scale benchmarks. It introduces the peak-end rule from psychology into this domain for the first time and proposes a lightweight, interpretable, and parameter-efficient general framework. Built upon a frozen Vision Transformer (ViT) backbone, the method leverages pretrained image aesthetic priors to identify salient and terminal frames, incorporates an aesthetic rhythm encoder to model temporal evolution, and employs a dynamic gating fusion mechanism to enhance cross-domain robustness. The approach achieves state-of-the-art performance on both VADB and DIVIDE-3K datasets, demonstrating the effectiveness of psychologically inspired modeling for video aesthetics.

Abstract

Video aesthetic assessment (VAA) aims to predict how aesthetically pleasing a video is, yet remains far less explored than other visual assessment tasks. Its progress is hindered not only by the scarcity of large-scale benchmarks, but also by the intrinsic subjectivity of aesthetic judgment, which is shaped by human perception. In this paper, we revisit VAA from a psychological perspective and propose \textit{Peak-End-Net}, a lightweight and interpretable framework inspired by the \textit{peak-end rule}, which suggests that people tend to judge a temporal experience mainly according to its salient moments and the ending. Building on this intuition, we first transfer knowledge from image aesthetic assessment (IAA) to VAA by introducing a pretrained IAA head to produce frame-wise aesthetic priors, which serve as surrogate signals for identifying aesthetically salient moments and guiding \textit{peak-end rule}-based temporal aggregation. To further capture how a video evolves aesthetically over time, we design an aesthetic rhythm encoder that models temporal progression beyond isolated moments. Additionally, we refine the overall assessment through a dynamic gated fusion mechanism to improve robustness under distribution shift. Our method is built on a frozen vision transformer (ViT) and requires only a small number of trainable parameters, making it scalable and parameter-efficient. Extensive experiments on two existing VAA benchmarks, including in-domain evaluation on VADB and cross-domain testing on DIVIDE-3K, demonstrate that our approach achieves state-of-the-art performance, affirming the value of psychologically grounded modeling for VAA. Our code and models are available at https://github.com/AMAP-ML/Peak-End-Net.

Problem

• video aesthetic assessment
• aesthetic judgment
• subjectivity
• temporal experience
• visual assessment

Innovation

• Peak-End Rule
• Video Aesthetic Assessment
• Aesthetic Rhythm Encoder
• Dynamic Gated Fusion
• Parameter-Efficient Learning