Lameness detection in dairy cows using pose estimation and bidirectional LSTMs

2025-08-14

AI Summary

This study addresses the challenge of early automatic lameness detection in dairy cattle. We propose an end-to-end framework that eliminates the need for manual annotation and handcrafted feature engineering. First, we employ the T-LEAP markerless pose estimation model to extract sequential keypoint trajectories from video. Subsequently, a bidirectional long short-term memory (BLSTM) network is designed to directly model spatiotemporal dynamics within ultra-short (1-second) keypoint sequences for binary lameness classification. By bypassing conventional reliance on domain-specific motion features and large-scale labeled datasets, our approach is particularly suited to low-data, short-sequence scenarios common in practical farm settings. Experimental results demonstrate that the optimal model achieves 85% classification accuracy—outperforming traditional handcrafted-feature methods by 5 percentage points—validating its potential for lightweight, efficient, and deployable implementation in real-world livestock farming environments.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This study presents a lameness detection approach that combines pose estimation and Bidirectional Long-Short-Term Memory (BLSTM) neural networks. Combining pose-estimation and BLSTMs classifier offers the following advantages: markerless pose-estimation, elimination of manual feature engineering by learning temporal motion features from the keypoint trajectories, and working with short sequences and small training datasets. Motion sequences of nine keypoints (located on the cows' hooves, head and back) were extracted from videos of walking cows with the T-LEAP pose estimation model. The trajectories of the keypoints were then used as an input to a BLSTM classifier that was trained to perform binary lameness classification. Our method significantly outperformed an established method that relied on manually-designed locomotion features: our best architecture achieved a classification accuracy of 85%, against 80% accuracy for the feature-based approach. Furthermore, we showed that our BLSTM classifier could detect lameness with as little as one second of video data.

Problem

• Detect lameness in dairy cows using pose estimation
• Classify lameness with BLSTMs and keypoint trajectories
• Improve accuracy over manual feature-based methods

Innovation

• Markerless pose-estimation for lameness detection
• BLSTM learns temporal motion features automatically
• Works with short video sequences effectively