Before you <think>, monitor: Implementing Flavell's metacognitive framework in LLMs

2025-10-18

AI Summary

Existing LLM inference methods suffer from structural deficiencies: Monitor-Generate lacks a verification feedback loop, while Generate-Verify omits pre-task assessment, leading to redundant iterations and low efficiency. This paper introduces the first systematic formalization of Flavell’s metacognitive theory into a three-stage iterative architecture—Plan (pre-task monitoring), Solve (generation), and Verify & Refine (validation and refinement)—enabling synergistic enhancement of strategic planning and result optimization. By incorporating proactive monitoring before generation, the framework significantly reduces unnecessary iterations. Evaluated on GSM8K, it achieves 75.42% accuracy, surpassing SELF-REFINE (68.44%) and Self-Verification (67.07%). The average number of solution attempts decreases from 2.0 to 1.3, with only a 27–37% increase in inference cost—demonstrating simultaneous gains in accuracy and computational efficiency.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Current approaches to enhancing LLM reasoning follows two isolated paradigms: Monitor-Generate methods like Plan-and-Solve (Wang et al., 2023) and SELF-DISCOVER (Zhou et al., 2024) excel at strategic planning but lack mechanisms to verify whether selected strategies succeed; while Generate-Verify approaches like Self-Verification (Weng et al., 2022) and SELF-REFINE (Madaan et al., 2023) iteratively refine outputs but commence generation blindly without task assessment. This separation creates inefficiencies -- strategies fail without feedback, and refinement occurs without strategic grounding. We address this gap by implementing Flavell's cognitive monitoring model (1979) from the broader Monitor-Generate-Verify framework (Oh and Gobet, 2025), operationalising it as a three-phase iterative system. On GSM8K, preliminary results show 75.42% accuracy versus 68.44% for SELF-REFINE and 67.07% for Self-Verification, while requiring fewer attempts (1.3 vs 2.0) at 27-37% increased inference cost. These initial findings suggest upfront monitoring produces higher-quality initial solutions that reduce refinement needs, though evaluation beyond arithmetic reasoning is needed to establish generalisability.

Problem

• Addressing inefficiencies in isolated LLM reasoning paradigms
• Implementing Flavell's metacognitive monitoring framework in LLMs
• Integrating strategic planning with verification mechanisms systematically

Innovation

• Implements Flavell's cognitive monitoring model
• Uses three-phase iterative Monitor-Generate-Verify system
• Combines strategic planning with verification mechanisms