Wandering around: A bioinspired approach to visual attention through object motion sensitivity

2025-02-10

AI Summary

To address the challenges of dynamic object detection and visual attention localization under event-camera motion, this paper proposes a biologically inspired spiking neural network (SNN) visual attention system. Methodologically, it integrates an event camera with a pan-tilt platform to emulate fixational eye movements, enabling an unsupervised, learning-free motion segmentation mechanism that accurately decouples object motion from camera ego-motion and achieves rapid foveal saccadic localization. Its key contribution is the first incorporation of fixational eye movements into an SNN framework, supporting event-driven real-time processing. Experiments demonstrate an average IoU of 82.2% and SSIM of 96% for multi-object motion segmentation; object detection accuracy reaches 89.8% under low-light conditions and 88.8% in office scenes; and the system achieves a dynamic response latency of only 0.12 seconds—exhibiting both real-time performance and cross-scenario robustness.

Technology Category

• Computer Vision: Motion & Tracking
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

Active vision enables dynamic visual perception, offering an alternative to static feedforward architectures in computer vision, which rely on large datasets and high computational resources. Biological selective attention mechanisms allow agents to focus on salient Regions of Interest (ROIs), reducing computational demand while maintaining real-time responsiveness. Event-based cameras, inspired by the mammalian retina, enhance this capability by capturing asynchronous scene changes enabling efficient low-latency processing. To distinguish moving objects while the event-based camera is in motion the agent requires an object motion segmentation mechanism to accurately detect targets and center them in the visual field (fovea). Integrating event-based sensors with neuromorphic algorithms represents a paradigm shift, using Spiking Neural Networks to parallelize computation and adapt to dynamic environments. This work presents a Spiking Convolutional Neural Network bioinspired attention system for selective attention through object motion sensitivity. The system generates events via fixational eye movements using a Dynamic Vision Sensor integrated into the Speck neuromorphic hardware, mounted on a Pan-Tilt unit, to identify the ROI and saccade toward it. The system, characterized using ideal gratings and benchmarked against the Event Camera Motion Segmentation Dataset, reaches a mean IoU of 82.2% and a mean SSIM of 96% in multi-object motion segmentation. The detection of salient objects reaches 88.8% accuracy in office scenarios and 89.8% in low-light conditions on the Event-Assisted Low-Light Video Object Segmentation Dataset. A real-time demonstrator shows the system's 0.12 s response to dynamic scenes. Its learning-free design ensures robustness across perceptual scenes, making it a reliable foundation for real-time robotic applications serving as a basis for more complex architectures.

Problem

• Develops bioinspired visual attention using motion sensitivity.
• Integrates event-based cameras with neuromorphic algorithms.
• Enhances real-time object detection in dynamic environments.

Innovation

• Uses event-based cameras for low-latency processing
• Implements Spiking Neural Networks for dynamic adaptation
• Integrates bioinspired attention system for motion segmentation