🤖 AI Summary
This study addresses the prevalence of shallow content in short-video recommendation systems driven by the attention economy. We propose the first Content Depth Score (CDS) and an evaluation benchmark, SCOPE-Bench, grounded in cognitive psychology. Utilizing a seven-level scale to annotate 150,000 videos, we evaluated 13 mainstream systems to reconstruct recommendation metrics from a cognitive perspective. Experiments reveal a pervasive bias toward superficial content, with deep content recommendation performance only marginally surpassing random baselines. By quantifying the cognitive value of video content, this work provides critical theoretical and empirical foundations for developing next-generation recommendation systems that balance user cognitive well-being with content quality.
📝 Abstract
Driven by the attention economy, short-video Recommender Systems (RSs) are primarily optimized to maximize user engagement by promoting videos that capture attention within seconds. These systems inherently favor shallow-content videos that are effective at attracting immediate attention. However, growing evidence suggests that prolonged exposure to such content may negatively affect users' cognitive engagement and mental well-being, raising concerns about the long-term societal impact of the short-video platform. To tackle this challenge, this paper introduces a new metric, the \textbf{Content Depth Score (CDS)}, to quantify the content depth of short videos. CDS measures the extent to which a video is expected to stimulate higher-order cognitive processes, using a seven-level scale grounded in established theories of cognitive psychology and learning. As an initial step toward this vision, we present \textbf{SCOPE-Bench}, the first benchmark for content-depth evaluation in short-video recommendation. Built upon a large-scale open-source short-video dataset, SCOPE-Bench provides CDS annotations for 150K videos, enabling systematic evaluation of RSs from a cognitive-content perspective. Leveraging SCOPE-Bench, we evaluate 13 representative RSs and reveal a consistent preference for shallow-content videos. Moreover, we find that these algorithms recommending cognitively deep content are only marginally better than random selection, highlighting a previously overlooked limitation of existing recommendation objectives. Our code and datasets are available at https://liweidengdavid.github.io/SCOPE-Bench/.