Robust Bidirectional Associative Memory via Regularization Inspired by the Subspace Rotation Algorithm

2025-11-14

AI Summary

Bidirectional associative memory (BAM) exhibits poor robustness under bidirectional backpropagation (B-BP) training, rendering it vulnerable to noise and adversarial attacks. To address this, we propose B-SRA—a novel gradient-free training algorithm inspired by subspace rotation—grounded in two core design principles: orthogonal weight initialization and gradient pattern alignment. By jointly enforcing orthogonal weight modification (OWM) and gradient pattern alignment (GPA) regularization, and conducting systematic ablation studies, we construct SAME, a highly robust BAM architecture. Experiments demonstrate that SAME consistently outperforms baseline methods across association capacities of 50–200 pattern pairs. It maintains stable recall performance under diverse noise corruptions and adversarial perturbations, achieving substantial improvements in robustness.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Bidirectional Associative Memory (BAM) trained with Bidirectional Backpropagation (B-BP) often suffers from poor robustness and high sensitivity to noise and adversarial attacks. To address these issues, we propose a novel gradient-free training algorithm, the Bidirectional Subspace Rotation Algorithm (B-SRA), which significantly improves the robustness and convergence behavior of BAM. Through comprehensive experiments, we identify two key principles -- orthogonal weight matrices (OWM) and gradient-pattern alignment (GPA) -- as central to enhancing the robustness of BAM. Motivated by these findings, we introduce new regularization strategies into B-BP, resulting in models with greatly improved resistance to corruption and adversarial perturbations. We further conduct an ablation study across different training strategies to determine the most robust configuration and evaluate BAM's performance under a variety of attack scenarios and memory capacities, including 50, 100, and 200 associative pairs. Among all methods, the SAME configuration, which integrates both OWM and GPA, achieves the strongest resilience. Overall, our results demonstrate that B-SRA and the proposed regularization strategies lead to substantially more robust associative memories and open new directions for building resilient neural architectures.

Problem

• Improving robustness of Bidirectional Associative Memory against noise
• Addressing sensitivity to adversarial attacks in neural networks
• Enhancing convergence behavior through novel regularization strategies

Innovation

• Proposed gradient-free Bidirectional Subspace Rotation Algorithm
• Introduced orthogonal weight matrices and gradient-pattern alignment
• Integrated both OWM and GPA in SAME configuration