Learning a General Model: Folding Clothing with Topological Dynamics

2025-04-29

AI Summary

Autonomous folding of high-degree-of-freedom, heavily self-occluding garments—such as jackets—remains challenging due to ambiguous state perception and complex, topology-varying deformations. Method: This paper proposes a topology-graph-based state representation and dynamics modeling framework. First, it constructs a low-dimensional topological graph driven by visible folding structures, integrating semantic segmentation and keypoint detection to explicitly encode deformation constraints and motion priors. Second, it introduces an enhanced graph neural network (GNN) that learns generalizable garment deformation dynamics and directly outputs the deformation Jacobian matrix required for control. Contribution/Results: To our knowledge, this is the first work to incorporate dynamic topological graphs into garment state modeling, eliminating reliance on complete geometric observations. Experiments on complex jacket-like garments demonstrate significant improvements in state estimation accuracy under severe self-occlusion, folding robustness, and cross-style generalization capability.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The high degrees of freedom and complex structure of garments present significant challenges for clothing manipulation. In this paper, we propose a general topological dynamics model to fold complex clothing. By utilizing the visible folding structure as the topological skeleton, we design a novel topological graph to represent the clothing state. This topological graph is low-dimensional and applied for complex clothing in various folding states. It indicates the constraints of clothing and enables predictions regarding clothing movement. To extract graphs from self-occlusion, we apply semantic segmentation to analyze the occlusion relationships and decompose the clothing structure. The decomposed structure is then combined with keypoint detection to generate the topological graph. To analyze the behavior of the topological graph, we employ an improved Graph Neural Network (GNN) to learn the general dynamics. The GNN model can predict the deformation of clothing and is employed to calculate the deformation Jacobi matrix for control. Experiments using jackets validate the algorithm's effectiveness to recognize and fold complex clothing with self-occlusion.

Problem

• Modeling complex clothing folding using topological dynamics
• Representing clothing states with low-dimensional topological graphs
• Predicting clothing deformation with improved Graph Neural Networks

Innovation

• Topological graph represents clothing state
• Semantic segmentation analyzes occlusion relationships
• Improved GNN learns general clothing dynamics