GitSearch: Enhancing Community Notes Generation with Gap-Informed Targeted Search

2026-02-09

AI Summary

This work addresses the challenges of insufficient fact-checking coverage and inconsistent quality in community content moderation under cold-start conditions. The authors propose GitSearch, a novel framework that, for the first time, leverages human-perceived information gaps—such as missing context—as a core signal to drive an end-to-end pipeline for targeted retrieval and compliant annotation generation. The approach employs a three-stage workflow: information gap identification, real-time web-based targeted retrieval, and synthesis of policy-compliant annotations. Evaluated on the PolBench benchmark, GitSearch achieves 99% coverage—nearly doubling that of the current state-of-the-art—and generates annotations that outperform human-written ones with a 69% win rate in helpfulness evaluations, attaining a helpfulness score of 3.87 compared to 3.36 for human annotations, thereby substantially mitigating cold-start limitations.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Search and Optimization: Learning to Search
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Community question answering

Abstract

Community-based moderation offers a scalable alternative to centralized fact-checking, yet it faces significant structural challenges, and existing AI-based methods fail in"cold start"scenarios. To tackle these challenges, we introduce GitSearch (Gap-Informed Targeted Search), a framework that treats human-perceived quality gaps, such as missing context, etc., as first-class signals. GitSearch has a three-stage pipeline: identifying information deficits, executing real-time targeted web-retrieval to resolve them, and synthesizing platform-compliant notes. To facilitate evaluation, we present PolBench, a benchmark of 78,698 U.S. political tweets with their associated Community Notes. We find GitSearch achieves 99% coverage, almost doubling coverage over the state-of-the-art. GitSearch surpasses human-authored helpful notes with a 69% win rate and superior helpfulness scores (3.87 vs. 3.36), demonstrating retrieval effectiveness that balanced the trade-off between scale and quality.

Problem

• Community Notes
• cold start
• information gaps
• fact-checking
• AI moderation

Innovation

• Gap-Informed Targeted Search
• Community Notes
• Information Gap
• Retrieval-Augmented Generation
• PolBench