Enhancing Layer Interaction Using Key-Correlated Layer Attention

2026-06-24

AI Summary

This work addresses the limitations of existing layer attention mechanisms—namely high computational complexity, static information updating, and inadequate modeling of long-range dependencies—by proposing Key-Correlated Layer Attention (KCLA). KCLA leverages the high cosine similarity among inter-layer Key representations to establish a dynamic cross-layer interaction mechanism with linear time complexity and constant space complexity. By integrating key-correlation-based linear attention with adaptive information fusion, KCLA maintains strong long-range dependency modeling capabilities while achieving computational efficiency independent of network depth. Experimental results demonstrate consistent and significant performance improvements across diverse tasks, including image classification, object detection, and medical image segmentation.

Abstract

Recent advances in network architecture design have introduced layer attention to enhance inter-layer interactions. In such frameworks, each layer queries all preceding layers to establish cross-layer connections. However, layer attention results in quadratic computational complexity with respect to network depth. To mitigate this issue, prior works have proposed Recurrent Layer Attention (RLA) and linear attention mechanisms, which suffer from static information updates and limited long-range cross-layer dependency modeling. To overcome these limitations, we propose Key-Correlated Layer Attention (KCLA), inspired by our observation that Key representations in layer attention exhibit high cosine similarity. KCLA achieves linear computational complexity while preserving dynamic information updates, directly derived from the foundational definition of layer attention. Furthermore, KCLA maintains long-range cross-layer connections and features a fixed spatial complexity, independent of network depth. Empirical evaluations demonstrate that KCLA delivers good performance across diverse tasks, including image recognition, object detection, and medical image segmentation. The code is publicly available at https://github.com/bgx666/KCLA.

Problem

• layer attention
• computational complexity
• cross-layer dependency
• network depth
• inter-layer interaction

Innovation

• Key-Correlated Layer Attention
• layer attention
• linear complexity
• cross-layer dependency
• dynamic information update