Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition

2021-11-16

IEEE Transactions on Pattern Analysis and Machine Intelligence

AI Summary

This study addresses the challenge of automatically selecting high-quality shots for movie trailers—a task where existing methods fall short of expert-level curation. We propose a semantic-graph-driven unsupervised shot selection method. First, we model a film as a shot semantic graph, integrating screenplay text and distilling narrative and affective semantics via joint contrastive learning. Second, we design an unsupervised graph traversal algorithm that dynamically selects high-potential shots based on narrative coherence and emotional tension. Our key contributions are: (1) a novel dual-task decomposition framework combining narrative structure recognition and emotion prediction; and (2) an interpretable, human-in-the-loop interactive shot selection mechanism. Experiments demonstrate that users can generate trailer segments in just 30 minutes whose quality matches expert-curated selections and significantly surpasses fully automatic baselines—thereby enhancing both creative efficiency and controllability.

Abstract

Movie trailers perform multiple functions: they introduce viewers to the story, convey the mood and artistic style of the film, and encourage audiences to see the movie. These diverse functions make trailer creation a challenging endeavor. In this work, we focus on finding trailer moments in a movie, i.e., shots that could be potentially included in a trailer. We decompose this task into two subtasks: narrative structure identification and sentiment prediction. We model movies as graphs, where nodes are shots and edges denote semantic relations between them. We learn these relations using joint contrastive training which distills rich textual information (e.g., characters, actions, situations) from screenplays. An unsupervised algorithm then traverses the graph and selects trailer moments from the movie that human judges prefer to ones selected by competitive supervised approaches. A main advantage of our algorithm is that it uses interpretable criteria, which allows us to deploy it in an interactive tool for trailer creation with a human in the loop. Our tool allows users to select trailer shots in under 30 minutes that are superior to fully automatic methods and comparable to (exclusive) manual selection by experts.

Problem

• Automatic Trailer Editing
• Machine Learning in Film
• AI Content Selection

Innovation

• Task Composition
• Graph Models
• Unsupervised Learning