Table of Links Abstract and 1. Introduction Preliminaries


Proposed Approach
3.1 Notation
3.2 Nueral Networks on SPD Manifolds
3.3 MLR in Structure Spaces
3.4 Neural Networks on Grassmann Manifolds


Experiments


Conclusion and References A. Notations B. MLR in Structure Spaces C. Formulation of MLR from the Perspective of Distances to Hyperplanes D. Human Action Recognition E. Node Classification F. Limitations of our work G. Some Related Definitions H. Computation of Canonical Representation I. Proof of Proposition 3.2 J. Proof of Proposition 3.4 K. Proof of Proposition 3.5 L. Proof of Proposition 3.6 M. Proof of Proposition 3.11 N. Proof of Proposition 3.12 2 PRELIMINARIES 2.1 SPD MANIFOLDS 2.2 GRASSMANN MANIFOLDS 2.3 NEURAL NETWORKS ON SPD AND GRASSMANN MANIFOLDS 2.3.1 NEURAL NETWORKS ON SPD MANIFOLDS The work in Huang & Gool (2017) introduces SPDNet with three novel layers, i.e., Bimap, LogEig, and ReEig layers that has become one of the most successful architectures in the field. In Brooks et al. (2019), the authors further improve SPDNet by developing Riemannian versions of batch normalization layers. Following these works, some works (Nguyen et al., 2019; Nguyen, 2021; Wang et al., 2021; Kobler et al., 2022; Ju & Guan, 2023) design variants of Bimap and batch normalization layers in SPD neural networks. The work in Chakraborty et al. (2020) presents a different approach based on intrinsic operations on SPD manifolds. Their proposed layers have nice theoretical properties. A common limitation of the above works is that they do not provide necessary mathematical tools for constructing many essential building blocks of DNNs on SPD manifolds. Recently, some works (Nguyen, 2022a;b; Nguyen & Yang, 2023) take a gyrovector space approach that enables natural generalizations of some building blocks of DNNs, e.g., MLR for SPD neural networks. 2.3.2 NEURAL NETWORKS ON GRASSMANN MANIFOLDS In Huang et al. (2018), the authors propose GrNet that explores the same rule of matrix backpropagation (Ionescu et al., 2015) as SPDNet. Some existing works (Wang & Wu, 2020; Souza et al.,2020) are also inspired by GrNet. Like their SPD counterparts, most existing Grassmann neural networks are not built upon a mathematical framework that allows one to generalize a broad class of DNNs to Grassmann manifolds. Using a gyrovector space approach, Nguyen & Yang (2023) has shown that some concepts in Euclidean spaces can be naturally extended to Grassmann manifolds. Authors:
(1) Xuan Son Nguyen, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (xuan-son.nguyen@ensea.fr);
(2) Shuo Yang, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (son.nguyen@ensea.fr);
(3) Aymeric Histace, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (aymeric.histace@ensea.fr). This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Preliminaries Proposed Approach
3.1 Notation
3.2 Nueral Networks on SPD Manifolds
3.3 MLR in Structure Spaces
3.4 Neural Networks on Grassmann Manifolds Experiments Conclusion and References Preliminaries Preliminaries Preliminaries Proposed Approach 3.1 Notation 3.2 Nueral Networks on SPD Manifolds 3.3 MLR in Structure Spaces 3.4 Neural Networks on Grassmann Manifolds Proposed Approach 3.1 Notation 3.1 Notation 3.2 Nueral Networks on SPD Manifolds 3.2 Nueral Networks on SPD Manifolds 3.3 MLR in Structure Spaces 3.3 MLR in Structure Spaces 3.4 Neural Networks on Grassmann Manifolds 3.4 Neural Networks on Grassmann Manifolds Experiments Experiments Experiments Conclusion and References Conclusion and References Conclusion and References A. Notations A. Notations B. MLR in Structure Spaces B. MLR in Structure Spaces C. Formulation of MLR from the Perspective of Distances to Hyperplanes C. Formulation of MLR from the Perspective of Distances to Hyperplanes D. Human Action Recognition D. Human Action Recognition E. Node Classification E. Node Classification F. Limitations of our work F. Limitations of our work G. Some Related Definitions G. Some Related Definitions H. Computation of Canonical Representation H. Computation of Canonical Representation I. Proof of Proposition 3.2 I. Proof of Proposition 3.2 J. Proof of Proposition 3.4 J. Proof of Proposition 3.4 K. Proof of Proposition 3.5 K. Proof of Proposition 3.5 L. Proof of Proposition 3.6 L. Proof of Proposition 3.6 M. Proof of Proposition 3.11 M. Proof of Proposition 3.11 N. Proof of Proposition 3.12 N. Proof of Proposition 3.12 2 PRELIMINARIES 2.1 SPD MANIFOLDS 2.1 SPD MANIFOLDS 2.2 GRASSMANN MANIFOLDS 2.3 NEURAL NETWORKS ON SPD AND GRASSMANN MANIFOLDS 2.3.1 NEURAL NETWORKS ON SPD MANIFOLDS 2.3.1 NEURAL NETWORKS ON SPD MANIFOLDS The work in Huang & Gool (2017) introduces SPDNet with three novel layers, i.e., Bimap, LogEig, and ReEig layers that has become one of the most successful architectures in the field. In Brooks et al. (2019), the authors further improve SPDNet by developing Riemannian versions of batch normalization layers. Following these works, some works (Nguyen et al., 2019; Nguyen, 2021; Wang et al., 2021; Kobler et al., 2022; Ju & Guan, 2023) design variants of Bimap and batch normalization layers in SPD neural networks. The work in Chakraborty et al. (2020) presents a different approach based on intrinsic operations on SPD manifolds. Their proposed layers have nice theoretical properties. A common limitation of the above works is that they do not provide necessary mathematical tools for constructing many essential building blocks of DNNs on SPD manifolds. Recently, some works (Nguyen, 2022a;b; Nguyen & Yang, 2023) take a gyrovector space approach that enables natural generalizations of some building blocks of DNNs, e.g., MLR for SPD neural networks. 2.3.2 NEURAL NETWORKS ON GRASSMANN MANIFOLDS 2.3.2 NEURAL NETWORKS ON GRASSMANN MANIFOLDS In Huang et al. (2018), the authors propose GrNet that explores the same rule of matrix backpropagation (Ionescu et al., 2015) as SPDNet. Some existing works (Wang & Wu, 2020; Souza et al.,2020) are also inspired by GrNet. Like their SPD counterparts, most existing Grassmann neural networks are not built upon a mathematical framework that allows one to generalize a broad class of DNNs to Grassmann manifolds. Using a gyrovector space approach, Nguyen & Yang (2023) has shown that some concepts in Euclidean spaces can be naturally extended to Grassmann manifolds. Authors: (1) Xuan Son Nguyen, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (xuan-son.nguyen@ensea.fr); (2) Shuo Yang, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (son.nguyen@ensea.fr); (3) Aymeric Histace, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (aymeric.histace@ensea.fr). Authors: Authors: (1) Xuan Son Nguyen, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (xuan-son.nguyen@ensea.fr); (2) Shuo Yang, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (son.nguyen@ensea.fr); (3) Aymeric Histace, ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, France (aymeric.histace@ensea.fr). This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Researchers Unlock Advanced Building Blocks for Neural Networks on Matrix Manifolds

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Comparative Performance Analysis of SymTax on Five Citation Recommendation Datasets

8 Deep Learning Best Practices I Learned About in 2017

An Introduction to “Liquid” Neural Networks

Building a Feedforward Neural Network from Scratch in Python

Building Machine Learning Models With TensorFlow

A Comparative Performance Analysis of SymTax on Five Citation Recommendation Datasets

8 Deep Learning Best Practices I Learned About in 2017

An Introduction to “Liquid” Neural Networks

Building a Feedforward Neural Network from Scratch in Python

Building Machine Learning Models With TensorFlow

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps