WARP-LCA: Efficient Convolutional Sparse Coding with Locally Competitive Algorithm

2024-10-24

Neurocomputing

AI Summary

Traditional Locally Competitive Algorithms (LCA) for Convolutional Sparse Coding (CSC) suffer from slow convergence, non-convex optimization induced by hard thresholding, and poor hardware efficiency. To address these issues, this paper proposes a hardware-cooperative LCA optimization framework. Our method introduces three key innovations: (1) a novel WARP-level parallel LCA solver that jointly accelerates sparse coding and dictionary update; (2) block-wise local response constraints and sliding-window sparse regularization to enhance biological plausibility and convergence stability; and (3) an adaptive dynamic thresholding mechanism to mitigate suboptimal solutions caused by hard thresholding. Evaluated on standard benchmarks including BSD500, our approach achieves a 5.3× speedup over conventional LCA, reduces memory footprint to one-quarter, improves reconstruction PSNR by 1.8 dB, and—critically—enables real-time video CSC for the first time.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Hardware-aware ML

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Problem

• Improves efficiency of convolutional sparse coding with LCA
• Addresses non-convex loss from hard-thresholding in LCA
• Enhances solution quality and convergence speed in LCA

Innovation

• Predictor network for LCA state initialization
• Combines LCA with predictive priming
• Enhances convergence speed and solution quality