Linking Heterogeneous Data with Coordinated Agent Flows for Social Media Analysis

2025-10-30

AI Summary

Existing automated methods struggle to unify modeling of heterogeneous social media data—including user behavior, textual content, temporal sequences, and network structures. Method: This paper proposes SIA, a collaborative multi-agent architecture grounded in large language models (LLMs). SIA employs a bottom-up insight taxonomy for analytical task planning, integrates a data coordinator to orchestrate multimodal data streams, and synergistically combines multimodal fusion, interpretable visualization, and interactive workflows to enable human-in-the-loop dynamic reasoning and validation. Contribution/Results: Experiments demonstrate that SIA significantly improves the diversity, accuracy, and interpretability of social insights—both in expert case studies and quantitative evaluations—while enhancing user trust. By enabling scalable, collaborative modeling of complex sociotechnical phenomena, SIA establishes a novel paradigm for human-AI co-analysis in social media intelligence.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Social media platforms generate massive volumes of heterogeneous data, capturing user behaviors, textual content, temporal dynamics, and network structures. Analyzing such data is crucial for understanding phenomena such as opinion dynamics, community formation, and information diffusion. However, discovering insights from this complex landscape is exploratory, conceptually challenging, and requires expertise in social media mining and visualization. Existing automated approaches, though increasingly leveraging large language models (LLMs), remain largely confined to structured tabular data and cannot adequately address the heterogeneity of social media analysis. We present SIA (Social Insight Agents), an LLM agent system that links heterogeneous multi-modal data -- including raw inputs (e.g., text, network, and behavioral data), intermediate outputs, mined analytical results, and visualization artifacts -- through coordinated agent flows. Guided by a bottom-up taxonomy that connects insight types with suitable mining and visualization techniques, SIA enables agents to plan and execute coherent analysis strategies. To ensure multi-modal integration, it incorporates a data coordinator that unifies tabular, textual, and network data into a consistent flow. Its interactive interface provides a transparent workflow where users can trace, validate, and refine the agent's reasoning, supporting both adaptability and trustworthiness. Through expert-centered case studies and quantitative evaluation, we show that SIA effectively discovers diverse and meaningful insights from social media while supporting human-agent collaboration in complex analytical tasks.

Problem

• Analyzing heterogeneous social media data including text, networks, and behaviors
• Overcoming limitations of existing automated approaches for multi-modal analysis
• Enabling coherent insight discovery through coordinated LLM agent workflows

Innovation

• Coordinated agent flows link heterogeneous multi-modal data
• Data coordinator unifies tabular textual network data
• Interactive interface enables traceable workflow for validation