Rethinking Encoder-Decoder Flow Through Shared Structures

2025-01-24

AI Summary

Conventional decoders for dense prediction tasks suffer from outdated architectural designs and insufficient cross-layer contextual sharing, limiting feature propagation efficiency and spatial consistency. Method: We propose a novel decoder architecture centered on a learnable, shared “bank”—a parameterized module dynamically resampled and fused across multiple scales to enable explicit cross-layer contextual reuse during decoding, thereby departing from traditional serial, layer-wise independent decoding paradigms. Built upon a Transformer backbone, the bank is jointly optimized end-to-end. Contribution/Results: Our approach significantly improves decoding efficiency and spatial coherence. On both natural and synthetic image depth estimation benchmarks, it substantially outperforms state-of-the-art methods, achieving superior accuracy and generalization under large-scale training. To our knowledge, this work presents the first systematic design and empirical validation of a universal, decoder-level contextual sharing mechanism.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Dense prediction tasks have enjoyed a growing complexity of encoder architectures, decoders, however, have remained largely the same. They rely on individual blocks decoding intermediate feature maps sequentially. We introduce banks, shared structures that are used by each decoding block to provide additional context in the decoding process. These structures, through applying them via resampling and feature fusion, improve performance on depth estimation for state-of-the-art transformer-based architectures on natural and synthetic images whilst training on large-scale datasets.

Problem

• Information Encoding
• Deep Learning
• Image Authentication

Innovation

• Bank Methodology
• Image Processing
• Information Encoding and Decoding