MultiCore+TPU Accelerated Multi-Modal TinyML for Livestock Behaviour Recognition

2025-04-10

AI Summary

To address the low efficiency and deployment challenges of livestock behavior recognition and motion tracking in remote pasture environments with constrained network connectivity, this paper proposes a lightweight multimodal TinyML system that fuses accelerometer and edge vision data to enable real-time on-device behavior classification, image classification, and object detection. We introduce a novel MultiCore+TPU co-acceleration deployment paradigm, supporting collaborative inference across multiple devices under offline or low-bandwidth conditions, and jointly optimize the model, hardware, and communication stack. Through quantization-aware compression and TPU acceleration, the model size is reduced by 270×, end-to-end inference latency drops below 80 ms, and accuracy matches state-of-the-art methods—while maintaining stable operation on commercial microcontroller units (MCUs). This significantly enhances the practicality and scalability of intelligent livestock monitoring systems.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The advancement of technology has revolutionised the agricultural industry, transitioning it from labour-intensive farming practices to automated, AI-powered management systems. In recent years, more intelligent livestock monitoring solutions have been proposed to enhance farming efficiency and productivity. This work presents a novel approach to animal activity recognition and movement tracking, leveraging tiny machine learning (TinyML) techniques, wireless communication framework, and microcontroller platforms to develop an efficient, cost-effective livestock sensing system. It collects and fuses accelerometer data and vision inputs to build a multi-modal network for three tasks: image classification, object detection, and behaviour recognition. The system is deployed and evaluated on commercial microcontrollers for real-time inference using embedded applications, demonstrating up to 270$ imes$ model size reduction, less than 80ms response latency, and on-par performance comparable to existing methods. The incorporation of the TinyML technique allows for seamless data transmission between devices, benefiting use cases in remote locations with poor Internet connectivity. This work delivers a robust, scalable IoT-edge livestock monitoring solution adaptable to diverse farming needs, offering flexibility for future extensions.

Problem

• Develops TinyML-based livestock behavior recognition system
• Fuses accelerometer and vision data for multi-modal tasks
• Enables real-time edge inference with low latency

Innovation

• Multi-modal TinyML for livestock behavior recognition
• Accelerometer and vision data fusion
• Real-time edge inference with low latency