Pest-Thinker: Learning to Think and Reason like Entomologists via Reinforcement Learning

2026-05-07

AI Summary

Agricultural pest identification is hindered by high inter-species complexity, substantial intra-species variation, and scarce annotated data, limiting the direct applicability of multimodal large language models (MLLMs). To address these challenges, this work constructs two high-quality image datasets, QFSD and AgriInsect, and introduces a morphology-oriented chain-of-thought (CoT) supervised fine-tuning approach. Furthermore, it proposes a novel knowledge-driven reinforcement learning framework that leverages an LLM-as-a-Judge to generate feature-based rewards and employs Group Relative Policy Optimization (GRPO) to guide the model toward attending to observable morphological evidence. This methodology significantly enhances the model’s fine-grained visual reasoning capabilities for both in-domain and out-of-domain pests, advancing agricultural pest recognition toward expert-level analytical performance.

Abstract

Pest-induced crop losses pose a major threat to global food security and sustainable agricultural development. While recent advances in Multimodal Large Language Models (MLLMs) have shown strong potential for visual understanding and smart agriculture, their direct application to pest recognition remains limited due to the domain's unique challenges such as high inter-species complexity, intra-species variability, and the scarcity of expert-annotated data. In this work, we introduce Pest-Thinker, a knowledge-driven reinforcement learning (RL) framework that enables MLLMs to reason over fine-grained pest morphology. We first construct two high-definition pest benchmarks, QFSD and AgriInsect, comprising diverse species and expert-annotated morphological traits. Leveraging these datasets, we synthesize Chain-of-Thought (CoT) reasoning trajectories to facilitate structured learning of pest-specific visual cues through Supervised Fine-Tuning (SFT). Subsequently, we employ Group Relative Policy Optimization (GRPO) with a novel feature reward that guides the model to focus on observable morphological evidence, assessed by an LLM-as-a-Judge strategy. Extensive experiments demonstrate that Pest-Thinker substantially improves both in-domain and out-of-domain morphological understanding, marking a step toward expert-level visual reasoning for intelligent agricultural pest analysis. The datasets and source code are available upon acceptance.

Problem

• pest recognition
• multimodal large language models
• morphological reasoning
• fine-grained visual understanding
• smart agriculture

Innovation

• Reinforcement Learning
• Multimodal Large Language Models
• Chain-of-Thought Reasoning
• Morphological Understanding
• Smart Agriculture