Examining the Utility of Self-disclosure Types for Modeling Annotators of Social Norms

2025-12-17

AI Summary

This study investigates which types of self-disclosure most effectively predict annotators’ judgments of social norms. Addressing demographic, attitudinal, relational, and experiential self-disclosure, we propose the first systematic typology and develop a personalized annotator prediction model via rule-based annotation, multivariate regression, and ablation analysis. Results show that demographic information yields the strongest predictive power; theory-driven categorization significantly outperforms data-driven automatic clustering; and performance saturates with only ≈5 task-relevant self-disclosure comments. Our core contributions are: (1) empirical validation of the proposed self-disclosure typology’s efficacy; (2) demonstration of a lightweight, interpretable approach to annotator modeling; and (3) provision of methodological foundations and empirical evidence for computational social norm modeling and human-AI collaborative annotation.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Application Domains: Humanities & Computational Social Science
• Natural Language Processing: Discourse, Pragmatics & Argument Mining

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

Recent work has explored the use of personal information in the form of persona sentences or self-disclosures to improve modeling of individual characteristics and prediction of annotator labels for subjective tasks. The volume of personal information has historically been restricted and thus little exploration has gone into understanding what kind of information is most informative for predicting annotator labels. In this work, we categorize self-disclosure sentences and use them to build annotator models for predicting judgments of social norms. We perform several ablations and analyses to examine the impact of the type of information on our ability to predict annotation patterns. We find that demographics are more impactful than attitudes, relationships, and experiences. Generally, theory-based approaches worked better than automatic clusters. Contrary to previous work, only a small number of related comments are needed. Lastly, having a more diverse sample of annotator self-disclosures leads to the best performance.

Problem

• Categorizing self-disclosure types to predict annotator labels
• Evaluating impact of information types on social norm judgments
• Determining most informative personal data for annotation modeling

Innovation

• Categorizing self-disclosure sentences for annotator modeling
• Finding demographics more impactful than attitudes or experiences
• Using theory-based approaches over automatic clustering methods