Video-based Automatic Lameness Detection of Dairy Cows using Pose Estimation and Multiple Locomotion Traits

2024-01-10

Computers and Electronics in Agriculture

AI Summary

This study addresses the challenge of automatic lameness detection in dairy cattle under outdoor conditions. We propose a pose-based gait analysis method integrating multidimensional kinematic features. Leveraging the T-LEAP model, we accurately extract trajectories of nine key anatomical landmarks from walking videos (99.6% detection accuracy) and systematically quantify six previously unexplored motion features—namely, back posture angle, head sway amplitude, inter-step distance, stride length, stance duration, and individual tracking displacement—thereby constructing a fused classifier. To enhance label reliability, we introduce a multi-observer scoring fusion strategy. Experimental results demonstrate that the six-feature ensemble achieves 80.1% classification accuracy, outperforming the best single-feature baseline by 3.5 percentage points. This improvement significantly enhances model robustness and clinical applicability, providing a practical, deployable technical framework for intelligent herd health monitoring in real-world pastoral settings.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Biometrics, Face, Gesture & Pose

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This study presents an automated lameness detection system that uses deep-learning image processing techniques to extract multiple locomotion traits associated with lameness. Using the T-LEAP pose estimation model, the motion of nine keypoints was extracted from videos of walking cows. The videos were recorded outdoors, with varying illumination conditions, and T-LEAP extracted 99.6% of correct keypoints. The trajectories of the keypoints were then used to compute six locomotion traits: back posture measurement, head bobbing, tracking distance, stride length, stance duration, and swing duration. The three most important traits were back posture measurement, head bobbing, and tracking distance. For the ground truth, we showed that a thoughtful merging of the scores of the observers could improve intra-observer reliability and agreement. We showed that including multiple locomotion traits improves the classification accuracy from 76.6% with only one trait to 79.9% with the three most important traits and to 80.1% with all six locomotion traits.

Problem

• Automated lameness detection in dairy cows using pose estimation
• Extracting multiple locomotion traits from cow walking videos
• Improving classification accuracy with multiple locomotion traits

Innovation

• Deep-learning image processing for lameness detection
• T-LEAP pose estimation for keypoint extraction
• Multiple locomotion traits improve classification accuracy