Annotating Dimensions of Social Perception in Text: The First Sentence-Level Dataset of Warmth and Competence

2026-01-09

arXiv.org

AI Summary

This study addresses the scarcity of sentence-level social cognition annotations in natural language processing research, which hinders the modeling of warmth—encompassing trust and sociability—and competence in textual content. To bridge this gap, the authors introduce W&C-Sent, the first sentence-level dataset annotated for warmth and competence, comprising over 1,600 English sentence–target pairs sourced from social media. Grounded in social psychological theory, each instance is meticulously labeled along three dimensions: trust, sociability, and competence. Data quality is ensured through crowdsourcing and rigorous multi-stage validation protocols. Benchmark evaluations using state-of-the-art large language models reveal significant limitations in their ability to recognize these nuanced social perception signals. This work thus provides a high-quality resource and a fine-grained foundation for interdisciplinary research at the intersection of computational social science and natural language processing.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Social Networks and Social Media: Computational social science
• Web Mining and Content Analysis: Sentiment analysis and opinion mining

Abstract

Warmth (W) (often further broken down into Trust (T) and Sociability (S)) and Competence (C) are central dimensions along which people evaluate individuals and social groups (Fiske, 2018). While these constructs are well established in social psychology, they are only starting to get attention in NLP research through word-level lexicons, which do not completely capture their contextual expression in larger text units and discourse. In this work, we introduce Warmth and Competence Sentences (W&C-Sent), the first sentence-level dataset annotated for warmth and competence. The dataset includes over 1,600 English sentence--target pairs annotated along three dimensions: trust and sociability (components of warmth), and competence. The sentences in W&C-Sent are from social media and often express attitudes and opinions about specific individuals or social groups (the targets of our annotations). We describe the data collection, annotation, and quality-control procedures in detail, and evaluate a range of large language models (LLMs) on their ability to identify trust, sociability, and competence in text. W&C-Sent provides a new resource for analyzing warmth and competence in language and supports future research at the intersection of NLP and computational social science.

Problem

• warmth
• competence
• social perception
• sentence-level annotation
• NLP

Innovation

• sentence-level annotation
• warmth and competence
• social perception
• computational social science
• large language models