ConceptCarve: Dynamic Realization of Evidence

2025-04-09

AI Summary

In large-scale social media, retrieving evidence linking abstract concepts (e.g., “Freedom”) to concrete behaviors (e.g., firearm possession) is challenging, and modeling their heterogeneous instantiations across communities remains difficult. Method: This paper proposes a retrieval-reasoning协同 framework that dynamically integrates BM25, dense passage retrieval (DPR), and large language model (LLM)-guided conceptual instantiation reasoning during retrieval to generate community-specific, interpretable evidence representations. Contribution/Results: It is the first work to embed LLMs directly into the retrieval pipeline, enabling dynamic search space characterization and community-adaptive deconstruction of conceptual semantics. Empirical evaluation across multiple communities demonstrates substantial gains over conventional sparse and dense retrieval baselines, effectively uncovering cognitive mechanisms and semantic evolution pathways underlying divergent conceptual expressions across demographic or ideological groups.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Finding evidence for human opinion and behavior at scale is a challenging task, often requiring an understanding of sophisticated thought patterns among vast online communities found on social media. For example, studying how gun ownership is related to the perception of Freedom, requires a retrieval system that can operate at scale over social media posts, while dealing with two key challenges: (1) identifying abstract concept instances, (2) which can be instantiated differently across different communities. To address these, we introduce ConceptCarve, an evidence retrieval framework that utilizes traditional retrievers and LLMs to dynamically characterize the search space during retrieval. Our experiments show that ConceptCarve surpasses traditional retrieval systems in finding evidence within a social media community. It also produces an interpretable representation of the evidence for that community, which we use to qualitatively analyze complex thought patterns that manifest differently across the communities.

Problem

• Finding evidence for human opinion and behavior at scale
• Identifying abstract concept instances in social media
• Handling different instantiations of concepts across communities

Innovation

• Combines traditional retrievers with LLMs
• Dynamically characterizes search space
• Produces interpretable evidence representation