Making Sense of Touch from the Child's View for Contrastive Learning

2026-06-30

AI Summary

This study investigates how infant touch facilitates visual concept learning and quantifies its underlying mechanisms. To this end, the authors constructed the first large-scale egocentric tactile event dataset from infants’ perspectives, comprising 264,000 two-second video clips, and developed a structured coding system for tactile behaviors. Integrating developmental psychology theory with a contrastive learning framework, they pretrained a computational model aligned with cognitive developmental principles. The results demonstrate that the model effectively simulates infant learning trajectories and corroborates the pivotal role of haptic experience in driving early visual concept formation. This work offers a novel pathway toward understanding the development of multimodal perception in infancy.

Abstract

Is the sense of touch a mechanism for human babies' learning of visual concepts? If so, can we quantify its importance, and to what extent do babies rely on their sense of touch for visual learning? To approach these questions in a principled way, we propose a structured coding system for baby-centric touch events, yielding a dataset of 264k two-second clips of touch events coded according to this system. Using this dataset, we pretrain developmentally grounded models that reveal promising insights into the nature of baby learning from touch.

Problem

• touch
• visual learning
• infant development
• sensory integration
• contrastive learning

Innovation

• baby-centric touch
• structured coding system
• developmentally grounded models
• contrastive learning
• multisensory learning