GPT and Prejudice: A Sparse Approach to Understanding Learned Representations in Large Language Models

2025-09-24

AI Summary

Large language models (LLMs) trained on uncurated corpora implicitly encode societal representations and biases, yet existing interpretability methods lack fine-grained semantic resolution for literary texts. Method: This work pioneers the systematic application of sparse autoencoders (SAEs) to literary-finetuned GPT architectures, constructing a Jane Austen–specific corpus and performing sparse decomposition of hidden-layer activations to yield interpretable, semantically grounded features. Contribution/Results: The approach successfully isolates sparse neural features strongly associated with gender roles, class structure, and social obligations—demonstrating LLM+SAE’s efficacy as a literary “corpus probe.” It introduces the first SAE-driven, scalable, and interpretable framework for bias detection in literary texts, establishing a novel paradigm for analyzing sociocultural representations in LLMs.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

As large language models (LLMs) are increasingly trained on massive, uncurated corpora, understanding both model representations and the data they internalize has become a major challenge. In this work, we show that pairing LLMs with sparse autoencoders (SAEs) enables interpretation not only of model behavior but also of the deeper structures, themes, and biases embedded in the training data. We train a GPT-style transformer model exclusively on the novels of Jane Austen, a corpus rich in social constructs and narrative patterns. We then apply SAEs to hidden states across multiple layers, uncovering sparse, interpretable features that reflect the key narratives and concepts present in the corpus, including gender, class, and societal duty. Our findings demonstrate that LLMs combined with SAEs can act as scalable probes into complex datasets, offering a new path for corpus exploration, bias discovery, and model interpretability at scale.

Problem

• Analyzing learned representations in large language models using sparse autoencoders
• Uncovering embedded social constructs and biases in training data
• Developing scalable methods for corpus exploration and model interpretability

Innovation

• Sparse autoencoders analyze GPT model representations
• SAEs extract interpretable features from hidden states
• Combining LLMs with SAEs enables scalable bias discovery