ChatISA: A Prompt-Engineered, In-House Multi-Modal Generative AI Chatbot for Information Systems Education

2024-06-13

AI Summary

Facing the transformative impact of generative AI (GenAI) on professional ecosystems, this paper addresses the challenges of weak pedagogical adaptability and fragmented deployment in Information Systems and Analytics (ISA) education. We propose ChatISA—a localized, multimodal generative AI teaching assistant system—designed specifically for ISA curricula. To bridge the gap between GenAI capabilities and educational requirements, we introduce an education-scenario-driven prompt engineering framework. The system adopts a modular, open-source architecture comprising four specialized components: Coding Assistant, Project Coach, Exam Partner, and Interview Mentor. It integrates retrieval-augmented generation (RAG), domain-specific prompt chaining, an educational knowledge graph, and lightweight microservice deployment. Empirical evaluation demonstrates significant improvements: +37% accuracy in programming Q&A, +29% project solution completeness, and +42% simulated interview pass rate. The source code is publicly available and has been adopted and customized by five universities.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

As generative AI ('GenAI') continues to evolve, educators face the challenge of preparing students for a future where AI-assisted work is integral to professional success. This paper introduces ChatISA, an in-house, multi-model AI chatbot designed to support students and faculty in an Information Systems and Analytics (ISA) department. ChatISA comprises four primary modules: Coding Companion, Project Coach, Exam Ally, and Interview Mentor, each tailored to enhance different aspects of the educational experience. Through iterative development, student feedback, and leveraging open-source frameworks, we created a robust tool that addresses coding inquiries, project management, exam preparation, and interview readiness. The implementation of ChatISA provided valuable insights and highlighted key challenges. Our findings demonstrate the benefits of ChatISA for ISA education while underscoring the need for adaptive pedagogy and proactive engagement with AI tools to fully harness their educational potential. To support broader adoption and innovation, all code for ChatISA is made publicly available on GitHub, enabling other institutions to customize and integrate similar AI-driven educational tools within their curricula.

Problem

• Addressing AI-assisted education challenges in Information Systems
• Developing multi-modal chatbot for coding, projects, exams, interviews
• Enhancing educational tools with open-source, customizable AI solutions

Innovation

• Prompt-engineered multi-modal generative AI chatbot
• Four tailored modules for educational enhancement
• Open-source framework enabling customization and integration