AdvisingWise: Supporting Academic Advising in Higher Educations Through a Human-in-the-Loop Multi-Agent Framework

2025-11-07

AI Summary

Higher education faces a critical imbalance between academic advising demand and supply, particularly during peak periods, where high student-to-advisor ratios cause significant response delays. To address this, we propose a human-AI collaborative multi-agent framework integrating large language models (LLMs), authoritative data source–driven retrieval, adaptive question-asking mechanisms, and a human-in-the-loop verification cycle—enabling automated yet human-audited response generation. Our key contributions are: (i) personalized question-answering grounded in institutional knowledge bases and dynamically refined through iterative clarification; and (ii) trustworthiness-by-design, wherein all AI-generated recommendations undergo real-time calibration via advisor feedback. Mixed-method evaluation demonstrates substantial improvements in response accuracy and personalization; advisor trust and adoption intent increase progressively over time. The system effectively alleviates peak-period advising bottlenecks and establishes a novel paradigm for deployable, trustworthy educational AI.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Academic advising is critical to student success in higher education, yet high student-to-advisor ratios limit advisors'capacity to provide timely support, particularly during peak periods. Recent advances in Large Language Models (LLMs) present opportunities to enhance the advising process. We present AdvisingWise, a multi-agent system that automates time-consuming tasks, such as information retrieval and response drafting, while preserving human oversight. AdvisingWise leverages authoritative institutional resources and adaptively prompts students about their academic backgrounds to generate reliable, personalized responses. All system responses undergo human advisor validation before delivery to students. We evaluate AdvisingWise through a mixed-methods approach: (1) expert evaluation on responses of 20 sample queries, (2) LLM-as-a-judge evaluation of the information retrieval strategy, and (3) a user study with 8 academic advisors to assess the system's practical utility. Our evaluation shows that AdvisingWise produces accurate, personalized responses. Advisors reported increasingly positive perceptions after using AdvisingWise, as their initial concerns about reliability and personalization diminished. We conclude by discussing the implications of human-AI synergy on the practice of academic advising.

Problem

• High student-to-advisor ratios limit timely academic support
• Academic advisors face capacity constraints during peak periods
• Need to automate time-consuming advising tasks while maintaining oversight

Innovation

• Multi-agent system automates academic advising tasks
• Human-in-the-loop validates all generated responses
• Adaptive prompting generates personalized student responses