Enhancing Higher Education with Generative AI: A Multimodal Approach for Personalised Learning

2025-02-11

AI Summary

This study addresses the low efficiency of personalized instruction and feedback analysis in higher education by proposing a multimodal generative AI teaching assistant tailored for undergraduate courses. Methodologically, it introduces a novel dual-engine architecture integrating ChatGPT and Google Bard to synergize textual interaction and visual understanding—enabling chart-to-code translation, fine-grained joint sentiment–emotion modeling, and automated teaching-evaluation summarization. The system integrates document parsing, multimodal sentiment/emotion recognition, a lightweight summarization model, and a web application framework. Experimental results demonstrate a 92% accuracy in student feedback sentiment classification, high completeness in extracting key evaluation indicators, and significantly improved response latency in instructor–student interactions. Deployed as a functional web-based teaching aid, the system establishes a reusable, multimodal collaborative paradigm for deploying large language models in educational contexts.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This research explores the opportunities of Generative AI (GenAI) in the realm of higher education through the design and development of a multimodal chatbot for an undergraduate course. Leveraging the ChatGPT API for nuanced text-based interactions and Google Bard for advanced image analysis and diagram-to-code conversions, we showcase the potential of GenAI in addressing a broad spectrum of educational queries. Additionally, the chatbot presents a file-based analyser designed for educators, offering deep insights into student feedback via sentiment and emotion analysis, and summarising course evaluations with key metrics. These combinations highlight the crucial role of multimodal conversational AI in enhancing teaching and learning processes, promising significant advancements in educational adaptability, engagement, and feedback analysis. By demonstrating a practical web application, this research underlines the imperative for integrating GenAI technologies to foster more dynamic and responsive educational environments, ultimately contributing to improved educational outcomes and pedagogical strategies.

Problem

• Enhancing higher education with Generative AI
• Developing a multimodal chatbot for personalized learning
• Integrating GenAI for dynamic educational environments

Innovation

• Multimodal chatbot for personalized learning
• ChatGPT API for text interactions
• Google Bard for image analysis