Tidiness Score-Guided Monte Carlo Tree Search for Visual Tabletop Rearrangement

2025-02-24

AI Summary

This work addresses two key challenges in tabletop tidying up (TTU): the absence of public benchmarks and the difficulty of defining target configurations for unseen objects. To this end, we propose a general-purpose, vision-only planning framework that does not require predefined “tidy” goal states. Methodologically, we introduce the first simulation-driven TTU dataset; design a generative tidiness score discriminator trained on RGB-D inputs in simulation and generalized to real-world images; and integrate Monte Carlo Tree Search (MCTS) for unsupervised, vision-guided rearrangement planning. Experiments demonstrate that our approach generates high-tidiness, diverse rearrangement trajectories across multiple real-world tabletop scenarios—including coffee tables and office desks—while exhibiting strong cross-scene generalization. The dataset and code are publicly released.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Planning

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In this paper, we present the tidiness score-guided Monte Carlo tree search (TSMCTS), a novel framework designed to address the tabletop tidying up problem using only an RGB-D camera. We address two major problems for tabletop tidying up problem: (1) the lack of public datasets and benchmarks, and (2) the difficulty of specifying the goal configuration of unseen objects. We address the former by presenting the tabletop tidying up (TTU) dataset, a structured dataset collected in simulation. Using this dataset, we train a vision-based discriminator capable of predicting the tidiness score. This discriminator can consistently evaluate the degree of tidiness across unseen configurations, including real-world scenes. Addressing the second problem, we employ Monte Carlo tree search (MCTS) to find tidying trajectories without specifying explicit goals. Instead of providing specific goals, we demonstrate that our MCTS-based planner can find diverse tidied configurations using the tidiness score as a guidance. Consequently, we propose TSMCTS, which integrates a tidiness discriminator with an MCTS-based tidying planner to find optimal tidied arrangements. TSMCTS has successfully demonstrated its capability across various environments, including coffee tables, dining tables, office desks, and bathrooms. The TTU dataset is available at: https://github.com/rllab-snu/TTU-Dataset.

Problem

• Develops a tidiness score-guided Monte Carlo tree search.
• Addresses lack of datasets and benchmarks for tabletop tidying.
• Solves goal configuration difficulty for unseen objects.

Innovation

• Tidiness score-guided Monte Carlo tree search
• Vision-based discriminator predicts tidiness score
• MCTS-based planner finds diverse tidied configurations