Multimodal Control of Manipulators: Coupling Kinematics and Vision for Self-Driving Laboratory Operations

2025-12-03

AI Summary

This study addresses the trajectory planning challenge for redundant robotic arms equipped with coupled-finger grippers in laboratory automation, where high-dimensional configuration spaces and task-constraint coupling impede conventional approaches. We propose an autonomous manipulation framework integrating visual perception with multimodal motion planning. Collision-free trajectories are generated using RRT*, while an accurate forward kinematic model is established via screw theory. Three inverse kinematics solvers—Jacobian transpose (JT), pseudoinverse (PI), and damped least squares (DLS)—are systematically compared. Innovatively, manipulability analysis and joint-space Jacobian computation are introduced to quantitatively evaluate trajectory smoothness, positional accuracy (RMSE), and higher-order motion continuity. Results demonstrate that DLS significantly enhances motion stability and end-effector precision, achieving superior robustness and adaptability in complex experimental environments. This work provides an efficient, reliable control strategy for autonomous robotic manipulation under stringent operational constraints.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Computer Vision: Motion & Tracking

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Motion planning schemes are used for planning motions of a manipulator from an initial pose to a final pose during a task execution. A motion planning scheme generally comprises of a trajectory planning method and an inverse kinematic solver to determine trajectories and joints solutions respectively. In this paper, 3 motion planning schemes developed based on Jacobian methods are implemented to traverse a redundant manipulator with a coupled finger gripper through given trajectories. RRT* algorithm is used for planning trajectories and screw theory based forward kinematic equations are solved for determining joint solutions of the manipulator and gripper. Inverse solutions are computed separately using 3 Jacobian based methods such as Jacobian Transpose (JT), Pseudo Inverse (PI), and Damped Least Square (DLS) methods. Space Jacobian and manipulability measurements of the manipulator and gripper are obtained using screw theory formulations. Smoothness and RMSE error of generated trajectories and velocity continuity, acceleration profile, jerk, and snap values of joint motions are analysed for determining an efficient motion planning method for a given task. Advantages and disadvantages of the proposed motion planning schemes mentioned above are analysed using simulation studies to determine a suitable inverse solution technique for the tasks.

Problem

• Develops motion planning schemes for redundant manipulators with grippers
• Compares Jacobian-based inverse kinematic methods for trajectory execution
• Analyzes performance metrics to determine efficient planning for laboratory tasks

Innovation

• Uses RRT* algorithm for trajectory planning in manipulators.
• Applies screw theory for forward kinematics and Jacobian methods.
• Compares Jacobian-based inverse solutions for motion efficiency.