KidneyTalk-open: No-code Deployment of a Private Large Language Model with Medical Documentation-Enhanced Knowledge Database for Kidney Disease

2025-03-06

AI Summary

To address the challenges of localization, privacy preservation, and clinical knowledge integration in nephrology decision-making, this paper introduces the first code-free, privacy-preserving desktop-scale large language model (LLM) system. Methodologically: (1) we develop a lightweight local inference framework enabling zero-barrier deployment of open-source LLMs (e.g., DeepSeek-R1, Qwen2.5); (2) we propose AddRep, an adaptive retrieval-augmented generation framework leveraging multi-agent collaboration to improve recall over structured medical documents; and (3) we design a clinician-oriented graphical interface for document management and question-answering. Evaluated on 1,455 nephrology-specific questions, our system achieves 29.1% accuracy (+8.1% over baseline) and exhibits strong hallucination mitigation, with a rejection rate of only 4.9%. It significantly outperforms existing desktop LLM tools—including AnythingLLM, Chatbox, and GPT4All—on both accuracy and reliability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Privacy-preserving medical decision support for kidney disease requires localized deployment of large language models (LLMs) while maintaining clinical reasoning capabilities. Current solutions face three challenges: 1) Cloud-based LLMs pose data security risks; 2) Local model deployment demands technical expertise; 3) General LLMs lack mechanisms to integrate medical knowledge. Retrieval-augmented systems also struggle with medical document processing and clinical usability. We developed KidneyTalk-open, a desktop system integrating three technical components: 1) No-code deployment of state-of-the-art (SOTA) open-source LLMs (such as DeepSeek-r1, Qwen2.5) via local inference engine; 2) Medical document processing pipeline combining context-aware chunking and intelligent filtering; 3) Adaptive Retrieval and Augmentation Pipeline (AddRep) employing agents collaboration for improving the recall rate of medical documents. A graphical interface was designed to enable clinicians to manage medical documents and conduct AI-powered consultations without technical expertise. Experimental validation on 1,455 challenging nephrology exam questions demonstrates AddRep's effectiveness: achieving 29.1% accuracy (+8.1% over baseline) with intelligent knowledge integration, while maintaining robustness through 4.9% rejection rate to suppress hallucinations. Comparative case studies with the mainstream products (AnythingLLM, Chatbox, GPT4ALL) demonstrate KidneyTalk-open's superior performance in real clinical query. KidneyTalk-open represents the first no-code medical LLM system enabling secure documentation-enhanced medical Q&A on desktop. Its designs establishes a new framework for privacy-sensitive clinical AI applications. The system significantly lowers technical barriers while improving evidence traceability, enabling more medical staff or patients to use SOTA open-source LLMs conveniently.

Problem

• Privacy-preserving medical decision support for kidney disease.
• No-code deployment of large language models for clinical use.
• Integration of medical knowledge into localized AI systems.

Innovation

• No-code deployment of SOTA open-source LLMs
• Medical document processing with intelligent filtering
• Adaptive Retrieval and Augmentation Pipeline (AddRep)