The Art of Audience Engagement: LLM-Based Thin-Slicing of Scientific Talks

2025-04-15

AI Summary

This study investigates the “thin-slicing” phenomenon in scientific talks—whether speech quality can be accurately predicted from extremely brief segments (e.g., the first 10% of a presentation). Leveraging transcribed texts from over one hundred real-world scientific talks, we develop an LLM-driven automated assessment framework and rigorously validate its reliability and validity using psychometric methods. Our work is the first to extend thin-slicing theory to public speaking contexts; it introduces a scalable, low-latency AI feedback paradigm for communication evaluation. Empirical results demonstrate high agreement between LLM-generated scores and human expert ratings (Pearson’s *r* > 0.85), and show that segments comprising less than 10% of total talk duration yield stable, robust predictions of overall quality. The framework provides an efficient, robust, and deployable assessment tool for AI-augmented communication training.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This paper examines the thin-slicing approach - the ability to make accurate judgments based on minimal information - in the context of scientific presentations. Drawing on research from nonverbal communication and personality psychology, we show that brief excerpts (thin slices) reliably predict overall presentation quality. Using a novel corpus of over one hundred real-life science talks, we employ Large Language Models (LLMs) to evaluate transcripts of full presentations and their thin slices. By correlating LLM-based evaluations of short excerpts with full-talk assessments, we determine how much information is needed for accurate predictions. Our results demonstrate that LLM-based evaluations align closely with human ratings, proving their validity, reliability, and efficiency. Critically, even very short excerpts (less than 10 percent of a talk) strongly predict overall evaluations. This suggests that the first moments of a presentation convey relevant information that is used in quality evaluations and can shape lasting impressions. The findings are robust across different LLMs and prompting strategies. This work extends thin-slicing research to public speaking and connects theories of impression formation to LLMs and current research on AI communication. We discuss implications for communication and social cognition research on message reception. Lastly, we suggest an LLM-based thin-slicing framework as a scalable feedback tool to enhance human communication.

Problem

• Predict presentation quality using brief excerpts (thin slices).
• Assess validity of LLM-based evaluations compared to human ratings.
• Develop scalable feedback tool for enhancing public speaking.

Innovation

• LLMs evaluate scientific talk transcripts
• Thin-slicing predicts presentation quality
• Short excerpts align with human ratings