Learning Characteristics of Reverse Quaternion Neural Network

2024-11-01

arXiv.org

AI Summary

To address the lack of systematic investigation into weight backpropagation mechanisms and rotational learning properties in multilayer feedforward quaternion neural networks (QNNs), this paper proposes the Reverse Quaternion Neural Network (RQNN). RQNN is the first model to systematically exploit the non-commutativity of quaternion multiplication to construct an inverse-weight propagation architecture, enabling more intrinsic modeling of rotation invariance. Theoretical analysis and empirical evaluation demonstrate that RQNN achieves learning speed comparable to state-of-the-art real-valued and quaternion models, while significantly outperforming them on rotation-generalization tasks. Its core contribution lies in revealing a representational gain mechanism—where non-commutativity enhances expressivity in inverse weight design—thereby establishing a novel paradigm for quaternion deep learning and providing an interpretable, principled approach to rotation-invariant representation learning.

Technology Category

• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Qualitative Reasoning
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The purpose of this paper is to propose a new multi-layer feedforward quaternion neural network model architecture, Reverse Quaternion Neural Network which utilizes the non-commutative nature of quaternion products, and to clarify its learning characteristics. While quaternion neural networks have been used in various fields, there has been no research report on the characteristics of multi-layer feedforward quaternion neural networks where weights are applied in the reverse direction. This paper investigates the learning characteristics of the Reverse Quaternion Neural Network from two perspectives: the learning speed and the generalization on rotation. As a result, it is found that the Reverse Quaternion Neural Network has a learning speed comparable to existing models and can obtain a different rotation representation from the existing models.

Problem

• Proposes Reverse Quaternion Neural Network architecture
• Investigates learning speed and rotation generalization
• Compares performance with existing quaternion models

Innovation

• Reverse direction weight application in quaternion networks
• Utilizes non-commutative quaternion product properties
• Investigates learning speed and rotation generalization