Problem
Research questions and friction points this paper is trying to address.
entity resolution
scientific literature
large language models
visual analytics
Metal-Organic Frameworks
Visual analysis of LLM-based entity resolution from scientific papers
📝 Abstract
This paper focuses on the visual analytics support for extracting domain-specific entity from extensive scientific literature, a task with inherent limitations using traditional named entity resolution methods. With the advent of large language models (LLMs) such as GPT-4, significant improvements over conventional machine learning approaches have been achieved due to LLM's capability on entity resolution integrate abilities such as understanding multiple types of text. This research introduces a new visual analysis pipeline that integrates these advanced LLMs with versatile visualization and interaction designs to support batch entity resolution. Specifically, we focus on a specific material science field of Metal-Organic Frameworks (MOFs) and a large data collection namely CSD-MOFs. Through collaboration with domain experts in material science, we obtain well-labeled synthesis paragraphs. We propose human-in-the-loop refinement over the entity resolution process using visual analytics techniques, which allows domain experts to interactively integrate insights into LLM intelligence, including error analysis and interpretation of the retrieval-augmented generation (RAG) algorithm. Our evaluation through the case study of example selection for RAG demonstrates that this human-machine collaborative approach improved single-document entity resolution accuracy by approximately 30%.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Large Language Models
Visual Analytics
Human-in-the-loop
Entity Resolution
Retrieval-Augmented Generation
🔎 Similar Papers
No similar papers found.
