MASS: Motion-Aligned Selective Scan for Refinement in Flow-Based Video Frame Interpolation

2026-06-26

AI Summary

This work addresses the challenge of correspondence ambiguity in video frame interpolation caused by large-scale nonlinear motion and complex occlusions. To this end, the authors propose a dynamic motion trajectory–based feature scanning mechanism that constructs feature sequences along nonlinear paths guided by optical flow. The method introduces a learnable residual velocity update and a velocity-aware state space model (SSM) to enable adaptive dense sampling and feature aggregation in fast-moving regions. Integrated with an end-to-end jointly optimized intermediate flow estimation and occlusion-aware refinement module, the proposed approach achieves state-of-the-art performance on standard benchmarks, particularly excelling in scenarios involving large displacements and intricate dynamic content.

Abstract

Video frame interpolation (VFI) remains a challenging task, particularly when dealing with large, non-linear motions and complex occlusions. While flow-based methods are prevalent, they often struggle with ambiguous correspondences. Recent VFI methods based on selective State Space Models (SSMs) are still limited by static grid-based scanning that misaligns with physical motion. In this paper, we propose Motion-Aligned Selective Scan (MASS), a novel framework that reformulates feature scanning from static spatial grids to dynamic motion trajectories. MASS builds a feature sequence along each pixel's flow-guided trajectory and aggregates it with an SSM. Specifically, we introduce a learnable non-linear path integration to approximate complex curved trajectories via residual velocity updates, and a velocity-aware SSM that dynamically adjusts the sampling budget and step size based on motion magnitude. This adaptive strategy allocates denser sampling to fast-motion regions while keeping static regions efficient. Furthermore, the aggregated states guide a refinement module to rectify intermediate flows and masks in an end-to-end manner. Extensive experiments indicate that MASS achieves highly competitive overall performance on standard benchmarks, establishing state-of-the-art results particularly in challenging scenarios with large displacements and complex dynamics.

Problem

• Video Frame Interpolation
• Motion Alignment
• Occlusion Handling
• Non-linear Motion
• Flow-based Methods

Innovation

• Motion-Aligned Scanning
• Selective State Space Model
• Flow-Guided Trajectory
• Adaptive Sampling
• Video Frame Interpolation