Beyond Text: Probing K-12 Educators' Perspectives and Ideas for Learning Opportunities Leveraging Multimodal Large Language Models

2025-07-28

AI Summary

This study investigates K–12 teachers’ perceptions, expectations, and practical challenges regarding the educational deployment of multimodal large language models (MLLMs). Method: Through participatory workshops, teachers co-designed pedagogical prototypes leveraging Claude 3.5’s Artifacts functionality—integrating textual and visual inputs to generate interactive learning tools. The study identifies two complementary application paradigms—“teacher-led” and “student-driven”—and analyzes divergent sensemaking pathways alongside core concerns regarding usability, fairness, and classroom integration. Contribution/Results: It provides the first systematic, practitioner-centered characterization of cognitive barriers and implementation requirements for MLLM adoption in education; delivers multiple empirically validated teaching prototypes; and distills actionable, experience-centered design principles and an implementation framework for next-generation educational technologies.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multimodal Large Language Models (MLLMs) are beginning to empower new user experiences that can flexibly generate content from a range of inputs, including images, text, speech, and video. These capabilities have the potential to enrich learning by enabling users to capture and interact with information using a variety of modalities, but little is known about how educators envision how MLLMs might shape the future of learning experiences, what challenges diverse teachers encounter when interpreting how these models work, and what practical needs should be considered for successful implementation in educational contexts. We investigated educator perspectives through formative workshops with 12 K-12 educators, where participants brainstormed learning opportunities, discussed practical concerns for effective use, and prototyped their own MLLM-powered learning applications using Claude 3.5 and its Artifacts feature for previewing code-based output. We use case studies to illustrate two contrasting end-user approaches (teacher-and student-driven), and share insights about opportunities and concerns expressed by our participants, ending with implications for leveraging MLLMs for future learning experiences.

Problem

• Exploring educators' visions for MLLMs in K-12 learning
• Identifying challenges teachers face with multimodal AI tools
• Addressing practical needs for MLLM implementation in education

Innovation

• Multimodal inputs for flexible content generation
• Formative workshops with educators for insights
• Prototyping MLLM-powered learning applications