FinTruthQA: A Benchmark Dataset for Evaluating the Quality of Financial Information Disclosure
The absence of automated tools for evaluating financial disclosure quality in Q&A forums on Chinese investor interaction platforms hinders regulatory oversight and market transparency. Method: We introduce FinDisclose-QA, the first benchmark for assessing financial disclosure quality in Chinese capital markets, comprising 6,000 real-world question-answer pairs manually annotated across four dimensions—completeness, accuracy, readability, and substantive relevance. We formally define and quantify multi-dimensional disclosure quality metrics specific to Q&A contexts and conduct multi-task evaluation using both traditional NLP models and large language models (LLMs) for question identification, answer relevance, readability, and substantive relevance assessment. Results: Experiments reveal that existing models perform well on question understanding but exhibit significant deficiencies in evaluating answer readability and substantive relevance. FinDisclose-QA provides a reproducible, extensible evaluation infrastructure for regtech applications, auditing practice, and academic research in financial communication.