Text-to-Image Generation for Vocabulary Learning Using the Keyword Method

2025-01-28

AI Summary

In foreign vocabulary acquisition, the traditional keyword method relies on unstable mental imagery, resulting in weak phonological–semantic associations and suboptimal retention. To address this, we propose a novel integration of the keyword method with text-to-image (T2I) generation, enabling the externalization of “mental visual associations”: learners articulate their internal imagery in natural language; optimized prompts are then fed into diffusion models (e.g., DALL·E 2, MidJourney) to generate concrete, personalized visual cues. We establish a reproducible pipeline—from imagery description to high-fidelity image synthesis—and validate it through user preference studies and controlled experiments. Results show DALL·E 2–generated images are most preferred, and the illustrated keyword condition yields significantly higher vocabulary retention than the text-only keyword condition (p < 0.01). This work introduces a scalable, multimodal intervention framework for lexical learning grounded in cognitive theory and generative AI.

Technology Category

• Natural Language Processing: Generation
• Computer Vision: Language and Vision
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The 'keyword method' is an effective technique for learning vocabulary of a foreign language. It involves creating a memorable visual link between what a word means and what its pronunciation in a foreign language sounds like in the learner's native language. However, these memorable visual links remain implicit in the people's mind and are not easy to remember for a large set of words. To enhance the memorisation and recall of the vocabulary, we developed an application that combines the keyword method with text-to-image generators to externalise the memorable visual links into visuals. These visuals represent additional stimuli during the memorisation process. To explore the effectiveness of this approach we first run a pilot study to investigate how difficult it is to externalise the descriptions of mental visualisations of memorable links, by asking participants to write them down. We used these descriptions as prompts for text-to-image generator (DALL-E2) to convert them into images and asked participants to select their favourites. Next, we compared different text-to-image generators (DALL-E2, Midjourney, Stable and Latent Diffusion) to evaluate the perceived quality of the generated images by each. Despite heterogeneous results, participants mostly preferred images generated by DALL-E2, which was used also for the final study. In this study, we investigated whether providing such images enhances the retention of vocabulary being learned, compared to the keyword method only. Our results indicate that people did not encounter difficulties describing their visualisations of memorable links and that providing corresponding images significantly improves memory retention.

Problem

• Foreign Language Learning
• Vocabulary Memory
• Pronunciation Association

Innovation

• DALL-E2
• Keyword Method
• Memory Enhancement