Alterbute: Editing Intrinsic Attributes of Objects in Images

2026-01-15

AI Summary

This work addresses the challenge of editing intrinsic object attributes—such as color, texture, material, and shape—in images while preserving object identity and scene context. To this end, the authors propose a novel diffusion-based method that integrates an identity reference image, textual prompts, a background image, and an object mask to jointly model intrinsic and extrinsic attribute variations during training. During inference, the extrinsic context is fixed to enable precise manipulation of intrinsic properties. Key innovations include a relaxed training objective and the introduction of Visual Named Entities (VNEs), which facilitate fine-grained identity-aware editing across diverse attribute configurations. Experimental results demonstrate that the proposed approach significantly outperforms existing methods, achieving high-fidelity, controllable, and multi-dimensional attribute editing without compromising object identity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We introduce Alterbute, a diffusion-based method for editing an object's intrinsic attributes in an image. We allow changing color, texture, material, and even the shape of an object, while preserving its perceived identity and scene context. Existing approaches either rely on unsupervised priors that often fail to preserve identity or use overly restrictive supervision that prevents meaningful intrinsic variations. Our method relies on: (i) a relaxed training objective that allows the model to change both intrinsic and extrinsic attributes conditioned on an identity reference image, a textual prompt describing the target intrinsic attributes, and a background image and object mask defining the extrinsic context. At inference, we restrict extrinsic changes by reusing the original background and object mask, thereby ensuring that only the desired intrinsic attributes are altered; (ii) Visual Named Entities (VNEs) - fine-grained visual identity categories (e.g.,''Porsche 911 Carrera'') that group objects sharing identity-defining features while allowing variation in intrinsic attributes. We use a vision-language model to automatically extract VNE labels and intrinsic attribute descriptions from a large public image dataset, enabling scalable, identity-preserving supervision. Alterbute outperforms existing methods on identity-preserving object intrinsic attribute editing.

Problem

• intrinsic attributes
• identity preservation
• object editing
• image manipulation
• visual identity

Innovation

• diffusion-based editing
• intrinsic attribute editing
• Visual Named Entities
• identity-preserving supervision
• context-aware image manipulation