Focus-LIME: Surgical Interpretation of Long-Context Large Language Models via Proxy-Based Neighborhood Selection

2026-02-04

AI Summary

This work addresses the challenge of attribution dilution in existing local model-agnostic explanation methods when applied to large language models with long contexts, where high-dimensional features hinder precise feature-level interpretability. To overcome this limitation, the authors propose Focus-LIME, a novel framework that introduces a surrogate model–guided coarse-to-fine explanation mechanism. It first leverages a surrogate model to identify an informative perturbation neighborhood and then performs context-aware perturbations and fine-grained feature attribution within the refined context. This approach effectively mitigates attribution dilution in long-context settings, significantly improving explanation fidelity across multiple benchmarks. Notably, Focus-LIME enables surgical-level precise explanations in long-context scenarios for the first time, establishing a practical foundation for deploying interpretable AI in high-stakes applications.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

As Large Language Models (LLMs) scale to handle massive context windows, achieving surgical feature-level interpretation is essential for high-stakes tasks like legal auditing and code debugging. However, existing local model-agnostic explanation methods face a critical dilemma in these scenarios: feature-based methods suffer from attribution dilution due to high feature dimensionality, thus failing to provide faithful explanations. In this paper, we propose Focus-LIME, a coarse-to-fine framework designed to restore the tractability of surgical interpretation. Focus-LIME utilizes a proxy model to curate the perturbation neighborhood, allowing the target model to perform fine-grained attribution exclusively within the optimized context. Empirical evaluations on long-context benchmarks demonstrate that our method makes surgical explanations practicable and provides faithful explanations to users.

Problem

• surgical interpretation
• long-context LLMs
• attribution dilution
• model-agnostic explanation
• feature-level explanation

Innovation

• Focus-LIME
• long-context LLMs
• surgical interpretation
• proxy-based neighborhood selection
• model-agnostic explanation