ATOM: Geometry-Aware Microgesture towards Object-Agnostic Tangible Interaction

2026-08-12

AI Summary

This work proposes a cross-object tangible interaction method for microgestures that operates without requiring predefined object models. By integrating fingertip sensing with generative 2D/3D geometric modeling, the system automatically extracts object geometry and leverages ergonomic principles to identify high-usability interaction regions, thereby transforming arbitrary handheld objects into tangible interfaces supporting 0D, 1D, and 2D microgestures. This study presents the first geometry-aware microgesture interaction framework independent of specific object identities and introduces an ergonomics-based mechanism for prioritizing interactive elements. Evaluations across ten everyday objects, including kitchen utensils, demonstrate that the system significantly outperforms ablation baselines in task completion rate, System Usability Scale (SUS) scores, and NASA-TLX cognitive workload metrics, confirming its strong generalization capability and practical utility.

Abstract

This paper presents ATOM, an integrated framework towards agnostic and tangible object interactions with microgestures. Our goal is to support microgesture interactions across different everyday objects, with the capability to automatically leverage the geometric affordance of each object. We formulate a fingertip-aware detection pipeline to leverage generative 2D and 3D models for geometry enhancement and refinement. We then introduce a usability-based method to prioritize the detected elements based on their ergonomic suitability for interactions. Building on this foundation, we further develop an AR system to transform everyday handheld objects into tangible user interfaces with 0D, 1D, and 2D microgesture interactions. Across transitions among everyday cooking objects of varying shapes and sizes, ATOM outperformed ablation baselines in task completion, usability (SUS), and workload (NASA-TLX). A further study with 10 objects demonstrates ATOM's generalizability across objects and grasps, highlighting its potential towards fluid, object-agnostic tangible interaction in real-world AR scenarios.

Problem

• microgesture
• object-agnostic
• tangible interaction
• geometry-aware
• AR

Innovation

• microgesture
• object-agnostic interaction
• geometry-aware detection
• tangible user interface
• generative 3D modeling