KnowThyself: An Agentic Assistant for LLM Interpretability

2025-11-05

AI Summary

Existing LLM interpretability tools are fragmented and require substantial coding expertise, hindering natural-language interaction and visualization-supported explanations. This paper proposes a multi-agent conversational interpretability platform that integrates heterogeneous explanation tools seamlessly into a unified dialogue workflow via a modular agent architecture—comprising query reformulation, intelligent routing, and context integration mechanisms. An orchestrating LLM coordinates agent routers and domain-specific analytical modules, enabling model upload, natural-language question answering, and interactive, visualization-augmented explanation generation. The key contribution is “zero-code” deep model introspection: users can perform comprehensive, interpretable analyses without programming, significantly lowering technical barriers. This enhances the usability, extensibility, and accessibility of interpretability tools, establishing a novel paradigm for building transparent and trustworthy AI systems. (149 words)

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Interpretability, Explainability, and Transparency
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We develop KnowThyself, an agentic assistant that advances large language model (LLM) interpretability. Existing tools provide useful insights but remain fragmented and code-intensive. KnowThyself consolidates these capabilities into a chat-based interface, where users can upload models, pose natural language questions, and obtain interactive visualizations with guided explanations. At its core, an orchestrator LLM first reformulates user queries, an agent router further directs them to specialized modules, and the outputs are finally contextualized into coherent explanations. This design lowers technical barriers and provides an extensible platform for LLM inspection. By embedding the whole process into a conversational workflow, KnowThyself offers a robust foundation for accessible LLM interpretability.

Problem

• Developing an agentic assistant for LLM interpretability
• Consolidating fragmented tools into chat-based interface
• Lowering technical barriers through conversational workflow

Innovation

• Chat-based interface for model uploads and queries
• Orchestrator LLM and agent router for query processing
• Interactive visualizations with guided contextual explanations