LiST: Local-Simplex Test-Time LoRA Fusion

2026-08-23

AI Summary

为解决静态LoRA适配器无法适应单个测试输入的问题，提出LiST框架，在测试时动态融合LoRA适配器，提高模型在未见任务上的鲁棒性。

Abstract

Task-specific LoRA adapters offer a modular way to specialize large language and vision-language models. However, existing adapter composition methods are mostly static and cannot adapt to individual test inputs. To address these issues, we propose \textbf{LiST}, a label-free test-time LoRA fusion framework that converts an existing LoRA bank into a target-conditioned local simplex and searches sample-specific fusion weights at inference time. LiST builds joint task representations from LoRA parameter anchors and prompt-level behavior vectors, retrieves neighboring adapters as a local search space, and performs branch-preserving fusion without updating the backbone or adapters. Candidate weights are selected by a prompt-level energy with prior, geometric, and stochastic-consistency constraints, and are deployed only when they pass a safe acceptance rule. Otherwise, LiST falls back to a target-conditioned prior. Experiments on multimodal and language benchmarks show that LiST outperforms static LoRA merging and conventional test-time adaptation baselines, while preserving task-specific adapter utility and improving robustness on unseen tasks.

Problem

• LoRA adapters
• test-time adaptation
• static composition

Innovation

• LiST
• test-time fusion
• LoRA adapters
• local simplex
• sample-specific weights