Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls

2026-05-04

AI Summary

This study addresses the challenging task of extracting key performance indicators (KPIs) from earnings call transcripts, which is hindered by the unstructured nature of the data, the absence of labeled examples, and significant domain divergence from existing SEC filings. To advance research in this area, the authors construct three benchmark datasets—SECB, ECB, and ECB-A—and introduce an expert-annotated subset to facilitate qualitative analysis. They propose the first human-validated, large language model (LLM)-driven open-ended KPI extraction framework, integrating a pretrained encoder with in-context learning strategies. Human evaluation demonstrates that the system achieves a precision of 79.7%, establishing a strong baseline for this task and substantially advancing the feasibility of consistent cross-domain KPI tracking.

Abstract

Earnings calls are a key source of financial information about public companies. However, extracting information from these calls is difficult. Unlike the templatic filings required by the U.S. Securities and Exchange Commission (SEC) to report a company's financial situation, earnings conference calls have no built-in labels, are unstructured, and feature conversational language. We explore this challenging domain by assessing the information captured by models trained on SEC filings and in-context learning methods. To establish a baseline, we first evaluate the generalization capabilities of SEC-trained models across established SEC datasets. To support our investigation, we introduce three novel benchmarks: (1) SEC Filings Benchmark (SECB), (2) Earnings Calls Benchmark (ECB), and ECB-A, a subset with 2,460 expert annotation groups to support our qualitative analysis. We find that encoder-based models struggle with the domain shift. Finally, we propose a system utilizing LLMs to perform open-ended extraction from unstructured call transcripts, verified by human evaluation (79.7% precision), providing a baseline for this valuable domain through the consistent tracking of emergent KPIs.

Problem

• KPI extraction
• earnings calls
• unstructured data
• financial information
• performance measurement

Innovation

• KPI extraction
• earnings calls
• large language models
• domain shift
• benchmark datasets