SA-LUT: Spatial Adaptive 4D Look-Up Table for Photorealistic Style Transfer

2025-06-16

AI Summary

Existing photorealistic style transfer methods face a fundamental trade-off: generative approaches achieve high fidelity but distort content structure and suffer from low efficiency, whereas lookup table (LUT)-based methods are computationally efficient yet lack local adaptability. This paper proposes Spatially Adaptive 4D Lookup Tables (SA-LUT), the first framework unifying the computational efficiency of LUTs with the local modeling capacity of neural networks. Key contributions include: (1) a style-guided 4D LUT generator; (2) a content-style cross-attention-driven contextual graph generator; and (3) PST50, the first dedicated benchmark for photorealistic style transfer. SA-LUT integrates multi-scale feature extraction, 4D table-based modeling, and context-aware spatial modulation. Experiments demonstrate a 66.7% reduction in LPIPS over conventional 3D LUTs and real-time video processing at 16 FPS—significantly outperforming state-of-the-art methods.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Photorealistic style transfer (PST) enables real-world color grading by adapting reference image colors while preserving content structure. Existing methods mainly follow either approaches: generation-based methods that prioritize stylistic fidelity at the cost of content integrity and efficiency, or global color transformation methods such as LUT, which preserve structure but lack local adaptability. To bridge this gap, we propose Spatial Adaptive 4D Look-Up Table (SA-LUT), combining LUT efficiency with neural network adaptability. SA-LUT features: (1) a Style-guided 4D LUT Generator that extracts multi-scale features from the style image to predict a 4D LUT, and (2) a Context Generator using content-style cross-attention to produce a context map. This context map enables spatially-adaptive adjustments, allowing our 4D LUT to apply precise color transformations while preserving structural integrity. To establish a rigorous evaluation framework for photorealistic style transfer, we introduce PST50, the first benchmark specifically designed for PST assessment. Experiments demonstrate that SA-LUT substantially outperforms state-of-the-art methods, achieving a 66.7% reduction in LPIPS score compared to 3D LUT approaches, while maintaining real-time performance at 16 FPS for video stylization. Our code and benchmark are available at https://github.com/Ry3nG/SA-LUT

Problem

• Bridges gap between style fidelity and content integrity in photorealistic style transfer
• Enhances LUT efficiency with neural network adaptability for precise color transformations
• Introduces PST50 benchmark for rigorous photorealistic style transfer evaluation

Innovation

• Style-guided 4D LUT Generator for color transformation
• Context Generator enables spatially-adaptive adjustments
• Combines LUT efficiency with neural network adaptability