Parallax Portrait Matting

2026-07-13

AI Summary

This work addresses the ill-posed nature of single-image portrait matting, which often leads to blurry details and color distortions when foreground and background textures are complex. To mitigate this, the authors propose a practical two-frame matting framework that requires only two images captured under slight viewpoint variations. The method leverages disparity to provide complementary geometric cues, jointly estimating a trimap and foreground/background motion. It further introduces an aligned view fusion strategy that combines direct background blending with a foreground cross-attention mechanism to effectively compensate for motion estimation errors. Experiments demonstrate that the approach significantly outperforms strong single-image baselines in complex scenes, recovering finer structural details and more accurate foreground colors—all without requiring specialized capture equipment.

Abstract

Image matting is highly ill-posed, especially when both the foreground and background are richly textured. While single-image matting methods learn strong priors from data, they often struggle on these challenging cases. Existing approaches improve results by requiring additional signals such as green screens, polarized lighting, or clean background images, but these typically rely on specialized capture setups. We present Parallax Portrait Matting, a practical two-frame matting method that uses a second image captured with slight viewpoint change. Such a setting arises naturally in burst photography, where small camera motion induces foreground-background parallax and provides complementary observations for matting. Our pipeline estimates trimaps and foreground/background motion, then constructs aligned views for prediction. To handle imperfect motion estimation, the network uses the background-aligned pair for direct fusion and the foreground-aligned cue through cross-attention for error compensation. Experiments show that our method recovers finer details and more accurate foreground colors than strong single-image matting baselines on challenging portrait cases.

Problem

• image matting
• parallax
• portrait
• foreground-background separation
• ill-posed problem

Innovation

• parallax
• image matting
• two-frame
• cross-attention
• motion alignment