SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation

2026-03-14

AI Summary

Existing native 3D generative models struggle to achieve precise control over three-dimensional structures, particularly in fine-grained details such as joints and poses. To address this limitation, this work proposes SK-Adapter—a lightweight structural adapter that, for the first time, treats 3D skeletons as first-class control signals. By injecting learnable skeleton-encoded tokens into a frozen 3D generative backbone, SK-Adapter enables high-fidelity global and local structural manipulation. The approach leverages a cross-attention mechanism to integrate joint coordinates with topological structure and is trained on a newly curated large-scale dataset, Objaverse-TMS, comprising 24k text-mesh-skeleton triplets. Experiments demonstrate that SK-Adapter significantly outperforms existing methods in structural controllability while preserving geometric and textural quality, thereby overcoming the ambiguity inherent in conventional text- or image-based prompts.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Native 3D generative models have achieved remarkable fidelity and speed, yet they suffer from a critical limitation: inability to prescribe precise structural articulations, where precise structural control within the native 3D space remains underexplored. This paper proposes SK-Adapter, a simple and yet highly efficient and effective framework that unlocks precise skeletal manipulation for native 3D generation. Moving beyond text or image prompts, which can be ambiguous for precise structure, we treat the 3D skeleton as a first-class control signal. SK-Adapter is a lightweight structural adapter network that encodes joint coordinates and topology into learnable tokens, which are injected into the frozen 3D generation backbone via cross-attention. This smart design allows the model to not only effectively "attend" to specific 3D structural constraints but also preserve its original generative priors. To bridge the data gap, we contribute Objaverse-TMS dataset, a large-scale dataset of 24k text-mesh-skeleton pairs. Extensive experiments confirm that our method achieves robust structural control while preserving the geometry and texture quality of the foundation model, significantly outperforming existing baselines. Furthermore, we extend this capability to local 3D editing, enabling the region specific editing of existing assets with skeletal guidance, which is unattainable by previous methods. Project Page: https://sk-adapter.github.io/

Problem

• 3D generation
• structural control
• skeletal manipulation
• native 3D space
• precise articulation

Innovation

• SK-Adapter
• skeleton-based control
• native 3D generation
• structural manipulation
• cross-attention injection