Self-Supervised Continuous Colormap Recovery from a 2D Scalar Field Visualization without a Legend

2025-07-28

AI Summary

This work addresses the inverse problem of recovering continuous color maps from 2D scalar field visualizations lacking color legends. We propose a self-supervised color mapping recovery method based on a disentanglement-reconstruction framework that jointly predicts both the underlying scalar field and the continuous color map—without any ground-truth annotations. Our approach parameterizes the color map using differentiable cubic B-splines and introduces two key losses: a color ordering loss to enforce perceptual consistency in hue progression, and a self-supervised reconstruction loss to ensure data-image fidelity. This enables joint optimization for color smoothness and correct ordinal relationships. Extensive experiments on synthetic and real-world datasets demonstrate high-fidelity recovery, enabling downstream tasks such as color adjustment and transfer. Moreover, our method generalizes robustly to diverse visualization styles—including those with legends or discrete palettes. To the best of our knowledge, this is the first fully unsupervised, end-to-end solution for continuous color map inversion.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Unsupervised & Self-Supervised Learning
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Web data visualization

Abstract

Recovering a continuous colormap from a single 2D scalar field visualization can be quite challenging, especially in the absence of a corresponding color legend. In this paper, we propose a novel colormap recovery approach that extracts the colormap from a color-encoded 2D scalar field visualization by simultaneously predicting the colormap and underlying data using a decoupling-and-reconstruction strategy. Our approach first separates the input visualization into colormap and data using a decoupling module, then reconstructs the visualization with a differentiable color-mapping module. To guide this process, we design a reconstruction loss between the input and reconstructed visualizations, which serves both as a constraint to ensure strong correlation between colormap and data during training, and as a self-supervised optimizer for fine-tuning the predicted colormap of unseen visualizations during inferencing. To ensure smoothness and correct color ordering in the extracted colormap, we introduce a compact colormap representation using cubic B-spline curves and an associated color order loss. We evaluate our method quantitatively and qualitatively on a synthetic dataset and a collection of real-world visualizations from the VIS30K dataset. Additionally, we demonstrate its utility in two prototype applications -- colormap adjustment and colormap transfer -- and explore its generalization to visualizations with color legends and ones encoded using discrete color palettes.

Problem

• Recover continuous colormap from 2D visualization without legend
• Decouple and reconstruct colormap and data simultaneously
• Ensure smoothness and correct color ordering in colormap

Innovation

• Decoupling-and-reconstruction strategy for colormap recovery
• Self-supervised optimizer with reconstruction loss
• Compact colormap representation using B-spline curves