Modeling Engagement with Brand and Organizational TikTok Videos Using Machine-Assisted Theory-Ensemble Annotation

2026-06-14

AI Summary

Short-form videos pose significant challenges for standardized modeling of user engagement due to their multimodal content and platform-specific algorithms. This study addresses this gap by computationally operationalizing classical interpretive theories from narratology, rhetoric, communication studies, and semiotics at scale. Leveraging a multimodal large language model, we automatically annotated 77 theory-driven structural variables across approximately 10,000 TikTok videos from Estonian brands and institutions, supplemented by human validation to assess reliability. Controlling for account size and video age, our model yielded a stable, albeit modest, improvement in predicting user engagement. Results indicate that variables related to perception and communication were reliably annotated, whereas deeper semiotic and archetypal structures proved more challenging to capture. This work establishes a systematic computational framework for analyzing the cultural structures embedded in short-form video content.

Abstract

Short-form video is difficult to study at scale because meaning emerges through audiovisual elements, language, and participatory, algorithmic and trend-based platform dynamics. Manual annotation of these layers is laborious at scale and difficult to standardize. We demonstrate how multimodal large language models (LLMs) can help address this bottleneck by annotating a set of 77 theory-driven structural variables derived from narratology, rhetoric, communication, and semiotics. We use this to explore content and estimate engagement with modest but consistent gains over account-size and video-age baselines in a corpus of about 10,000 TikTok videos of brand and organizational accounts from Estonia (covering a substantial share of the small country ecosystem). Human validation shows a reliability gradient: perceptual and communicative variables can be coded fairly reliably, while deeper semiotic and archetypal constructs are more difficult for both humans and machines. This approach of computational operationalization of long-standing interpretive theories can support several aims: exploratory cultural analytics of variation in short-form video culture, predictive modeling of platform dynamics, engagement, and audience feedback; and diagnostics for content creators to support choosing between structural and narrative strategies. Most annotated variables were not associated with platform success, as expected; the value of LLMs in this setting lies in making it feasible to assess large batteries of theoretically motivated variables, so that the subset carrying signal can be identified and translated into creator-facing guidance for a given niche.

Problem

• short-form video
• multimodal annotation
• engagement modeling
• TikTok
• computational content analysis

Innovation

• multimodal LLMs
• theory-driven annotation
• short-form video analytics
• computational semiotics
• engagement prediction