Audio-Visual Class-Incremental Learning for Fish Feeding intensity Assessment in Aquaculture

2025-04-21

AI Summary

To address catastrophic forgetting and data scarcity in Fish Feeding Intensity Assessment (FFIA) for cross-species continual learning in industrial aquaculture, this paper introduces AV-CIL-FFIA—the first audio-visual class-incremental learning benchmark for FFIA—and proposes HAIL-FFIA, a sample-free, prototype-driven framework. Methodologically, HAIL-FFIA establishes the first FFIA-specific audio-visual continual learning paradigm; designs a hierarchical representation with dual-path knowledge distillation to decouple general feeding patterns from species-specific features; and incorporates a dynamic modality balancing strategy to adaptively fuse multi-stage feeding signals. Experiments on AV-CIL-FFIA demonstrate that HAIL-FFIA significantly outperforms unimodal and state-of-the-art incremental learning methods, achieving substantial accuracy gains while imposing zero parameter or sample storage overhead—effectively mitigating cross-species catastrophic forgetting.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Fish Feeding Intensity Assessment (FFIA) is crucial in industrial aquaculture management. Recent multi-modal approaches have shown promise in improving FFIA robustness and efficiency. However, these methods face significant challenges when adapting to new fish species or environments due to catastrophic forgetting and the lack of suitable datasets. To address these limitations, we first introduce AV-CIL-FFIA, a new dataset comprising 81,932 labelled audio-visual clips capturing feeding intensities across six different fish species in real aquaculture environments. Then, we pioneer audio-visual class incremental learning (CIL) for FFIA and demonstrate through benchmarking on AV-CIL-FFIA that it significantly outperforms single-modality methods. Existing CIL methods rely heavily on historical data. Exemplar-based approaches store raw samples, creating storage challenges, while exemplar-free methods avoid data storage but struggle to distinguish subtle feeding intensity variations across different fish species. To overcome these limitations, we introduce HAIL-FFIA, a novel audio-visual class-incremental learning framework that bridges this gap with a prototype-based approach that achieves exemplar-free efficiency while preserving essential knowledge through compact feature representations. Specifically, HAIL-FFIA employs hierarchical representation learning with a dual-path knowledge preservation mechanism that separates general intensity knowledge from fish-specific characteristics. Additionally, it features a dynamic modality balancing system that adaptively adjusts the importance of audio versus visual information based on feeding behaviour stages. Experimental results show that HAIL-FFIA is superior to SOTA methods on AV-CIL-FFIA, achieving higher accuracy with lower storage needs while effectively mitigating catastrophic forgetting in incremental fish species learning.

Problem

• Addresses catastrophic forgetting in fish feeding intensity assessment
• Introduces AV-CIL-FFIA dataset for multi-species aquaculture environments
• Proposes HAIL-FFIA framework for efficient exemplar-free incremental learning

Innovation

• Introduces AV-CIL-FFIA dataset for fish feeding assessment
• Proposes HAIL-FFIA framework for audio-visual class-incremental learning
• Uses hierarchical representation and dynamic modality balancing