SS-GEN: A Social Story Generation Framework with Large Language Models

2024-06-22

AAAI Conference on Artificial Intelligence

AI Summary

Children with Autism Spectrum Disorder (ASD) face significant challenges in social understanding, yet conventional manual authoring of social stories suffers from high labor costs, limited personalization, and delayed responsiveness. Method: We propose StarSow—a hierarchical constrained prompting strategy—to construct the first high-quality LLM-generated social story dataset and a multidimensional clinical compliance evaluation framework. Leveraging constraint-driven prompt engineering, human-in-the-loop filtering, and supervised fine-tuning of small open-weight models (e.g., Llama-2, Qwen), StarSow replaces costly large-model APIs. Contribution/Results: Our approach achieves expert-level generation quality while strictly adhering to clinical guidelines, reduces deployment cost by 90%, and compresses inference prompts by 70%. To our knowledge, this is the first AI framework enabling scalable, low-cost, and real-time personalized social story generation for ASD intervention.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Children with Autism Spectrum Disorder (ASD) often misunderstand social situations and struggle to participate in daily routines. Social Stories™ are traditionally crafted by psychology experts under strict constraints to address these challenges but are costly and limited in diversity. As Large Language Models (LLMs) advance, there's an opportunity to develop more automated, affordable, and accessible methods to generate Social Stories in real-time with broad coverage. However, adapting LLMs to meet the unique and strict constraints of Social Stories is a challenging issue. To this end, we propose SS-GEN, a Social Story GENeration framework with LLMs. Firstly, we develop a constraint-driven sophisticated strategy named StarSow to hierarchically prompt LLMs to generate Social Stories at scale, followed by rigorous human filtering to build a high-quality dataset. Additionally, we introduce quality assessment criteria to evaluate the effectiveness of these generated stories. Considering that powerful closed-source large models require very complex instructions and expensive API fees, we finally fine-tune smaller language models with our curated high-quality dataset, achieving comparable results at lower costs and with simpler instruction and deployment. This work marks a significant step in leveraging AI to personalize Social Stories cost-effectively for autistic children at scale, which we hope can encourage future research on special groups.

Problem

• Automate Social Story generation for ASD children
• Reduce cost and increase diversity of Social Stories
• Adapt LLMs to strict Social Story constraints

Innovation

• Hierarchical prompting with StarSow for scalable generation
• Fine-tuning smaller models for cost-effective deployment
• Rigorous human filtering to ensure high-quality datasets