LumiNote: LLM-Assisted Multimodal Instruction for VR Stage Lighting Education

2026-09-15

AI Summary

LumiNote利用LLM辅助VR系统解决舞台灯光教育中抽象概念与实际操作结合的问题，通过转换教学意图成可执行演示和语言支持来辅助教学。

Abstract

Stage lighting education requires instructors to bridge abstract concepts, technical operations, and learner-understandable representations. While Virtual Reality (VR) removes physical constraints, existing systems provide limited support for live instruction. We present LumiNote, an LLM-assisted VR system that transforms spoken pedagogical intent into instructor-reviewable spatial annotations, executable demonstrations, and linguistic support. In an exploratory study with 3 instructors and 24 students, we examined how instructors incorporated LumiNote into familiar lighting topics and how students received the resulting representations. We found LLM assistance most valuable for expressive, under-specified goals, but requiring greater expert intervention for fixture-specific or spatial configuration requests. Instructors engaged with generated suggestions as a controllable refinement process, shifting effort from manual setup toward pedagogical expression. However, representations that externalized expert reasoning did not always align with novice comprehension. These findings characterize LLM-assisted VR instruction as a domain-grounded mediation process among expert expression, executable operations, and learner-facing representations.

Problem

• Stage Lighting Education
• Virtual Reality (VR)
• Live Instruction

Innovation

• LLM-assisted
• VR system
• spatial annotations
• executable demonstrations
• pedagogical expression