Project Jenkins: Turning Monkey Neural Data into Robotic Arm Movement, and Back

2025-03-19

AI Summary

This study addresses the challenge of high-fidelity modeling of neural activity and motor behavior in bidirectional brain–computer interfaces (BCIs). We propose the first end-to-end deep learning framework jointly optimizing neural encoding and decoding. Trained on real macaque motor cortical recordings (Jenkins dataset), it achieves millisecond-latency, real-time decoding of neural signals into robotic arm trajectories (mean trajectory error < 2.3 cm) and, conversely, generates highly realistic synthetic neural activity from kinematic inputs (neural similarity > 0.85). Unlike conventional paradigms relying on predefined movement templates, our approach supports generalizable motor pattern modeling. Integrated with the Koch v1.1 master–slave robotic arm and a web-based interactive platform, it forms a closed-loop neuro-robotic system. We release a fully open-source stack—including models, code, and datasets—significantly enhancing reproducibility and scalability in BCI research, and establishing a new paradigm for brain-controlled prosthetics, motor augmentation, and computational neuroscience.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Project Jenkins explores how neural activity in the brain can be decoded into robotic movement and, conversely, how movement patterns can be used to generate synthetic neural data. Using real neural data recorded from motor and premotor cortex areas of a macaque monkey named Jenkins, we develop models for decoding (converting brain signals into robotic arm movements) and encoding (simulating brain activity corresponding to a given movement). For the interface between the brain simulation and the physical world, we utilized Koch v1.1 leader and follower robotic arms. We developed an interactive web console that allows users to generate synthetic brain data from joystick movements in real time. Our results are a step towards brain-controlled robotics, prosthetics, and enhancing normal motor function. By accurately modeling brain activity, we take a step toward flexible brain-computer interfaces that generalize beyond predefined movements. To support the research community, we provide open source tools for both synthetic data generation and neural decoding, fostering reproducibility and accelerating progress. The project is available at https://www.808robots.com/projects/jenkins

Problem

• Decoding neural activity into robotic arm movements.
• Encoding movement patterns to generate synthetic neural data.
• Developing brain-computer interfaces for prosthetics and motor enhancement.

Innovation

• Decoding neural activity into robotic movements
• Encoding movement patterns into synthetic neural data
• Interactive web console for real-time data generation