Table of Links Abstract and 1 Introduction 2 Preliminaries 3. Revisiting Normalization 3.1 Revisiting Euclidean Normalization 3.2 Revisiting Existing RBN 4 Riemannian Normalization on Lie Groups 5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds 5.2 LieBN on SPD Manifolds 6 Experiments 6.1 Experimental Results 7 Conclusions, Acknowledgments, and References APPENDIX CONTENTS A Notations B Basic layes in SPDnet and TSMNet C Statistical Results of Scaling in the LieBN D LieBN as a Natural Generalization of Euclidean BN E Domain-specific Momentum LieBN for EEG Classification F Backpropagation of Matrix Functions G Additional Details and Experiments of LieBN on SPD manifolds H Preliminary Experiments on Rotation Matrices I Proofs of the Lemmas and Theories in the Main Paper 4 RIEMANNIAN NORMALIZATION ON LIE GROUPS In this section, the neutral element in the Lie group M is denoted as E. Notably, the neutral element E is not necessarily the identity matrix. We first clarify the essential properties of Euclidean BN and then present our normalization method, tailored for Lie groups. Two key points regarding Euclidean BN, as expressed by Eq. (6), are worth highlighting: (a) The standard BN (Ioffe & Szegedy, 2015) implicitly assumes a Gaussian distribution and can effectively normalize and transform the latent Gaussian distribution; (b) The centering and biasing operations control the mean, while the scaling controls the variance. Therefore, extending BN into Lie groups requires defining Gaussian distribution, centering, biasing, and scaling on Lie groups. Remark 4.3. The MLE of the mean of the Gaussian distribution in Eq. (12) have been examined in several previous works (Said et al., 2017; Chakraborty & Vemuri, 2019; Chakraborty, 2020). However, these studies primarily focus on particular manifolds or specific metrics. In contrast, our contribution lies in presenting a universally applicable result for all Lie groups with left-invariant metrics. While Eq. (12) briefly appeared in Kobler et al. (2022b), the authors only focus on SPD manifolds under AIM. The transformation of the population under their proposed RBN remains unexplored as well. Besides, while Chakraborty (2020) analyzed the population properties for their RBN over matrix Lie groups, their results were confined within a specific distance. In contrast, our work provides a more extensive examination, encompassing both population and sample properties of our LieBN in a general manner. Besides, all the discussion about our LieBN can be readily transferred to right-invariant metrics. This paper focuses on LieBN based on left-invariant metrics. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. [3] When M corresponds to R equipped with the standard Euclidean metric, Eq. (12) reduces to the Euclidean Gaussian distribution Authors:
(1) Ziheng Chen, University of Trento;
(2) Yue Song, University of Trento and a Corresponding author;
(3) Yunmei Liu, University of Louisville;
(4) Nicu Sebe, University of Trento. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3. Revisiting Normalization 3.1 Revisiting Euclidean Normalization 3.1 Revisiting Euclidean Normalization 3.2 Revisiting Existing RBN 3.2 Revisiting Existing RBN 4 Riemannian Normalization on Lie Groups 4 Riemannian Normalization on Lie Groups 5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds 5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds 5.2 LieBN on SPD Manifolds 5.2 LieBN on SPD Manifolds 6 Experiments 6 Experiments 6.1 Experimental Results 6.1 Experimental Results 7 Conclusions, Acknowledgments, and References 7 Conclusions, Acknowledgments, and References APPENDIX CONTENTS APPENDIX CONTENTS A Notations A Notations B Basic layes in SPDnet and TSMNet B Basic layes in SPDnet and TSMNet C Statistical Results of Scaling in the LieBN C Statistical Results of Scaling in the LieBN D LieBN as a Natural Generalization of Euclidean BN D LieBN as a Natural Generalization of Euclidean BN E Domain-specific Momentum LieBN for EEG Classification E Domain-specific Momentum LieBN for EEG Classification F Backpropagation of Matrix Functions F Backpropagation of Matrix Functions G Additional Details and Experiments of LieBN on SPD manifolds G Additional Details and Experiments of LieBN on SPD manifolds H Preliminary Experiments on Rotation Matrices H Preliminary Experiments on Rotation Matrices I Proofs of the Lemmas and Theories in the Main Paper I Proofs of the Lemmas and Theories in the Main Paper 4 RIEMANNIAN NORMALIZATION ON LIE GROUPS In this section, the neutral element in the Lie group M is denoted as E. Notably, the neutral element E is not necessarily the identity matrix. We first clarify the essential properties of Euclidean BN and then present our normalization method, tailored for Lie groups. Two key points regarding Euclidean BN, as expressed by Eq. (6), are worth highlighting: (a) The standard BN (Ioffe & Szegedy, 2015) implicitly assumes a Gaussian distribution and can effectively normalize and transform the latent Gaussian distribution; (b) The centering and biasing operations control the mean, while the scaling controls the variance. Therefore, extending BN into Lie groups requires defining Gaussian distribution, centering, biasing, and scaling on Lie groups. Remark 4.3. The MLE of the mean of the Gaussian distribution in Eq. (12) have been examined in several previous works (Said et al., 2017; Chakraborty & Vemuri, 2019; Chakraborty, 2020). However, these studies primarily focus on particular manifolds or specific metrics. In contrast, our contribution lies in presenting a universally applicable result for all Lie groups with left-invariant metrics. While Eq. (12) briefly appeared in Kobler et al. (2022b), the authors only focus on SPD manifolds under AIM. The transformation of the population under their proposed RBN remains unexplored as well. Besides, while Chakraborty (2020) analyzed the population properties for their RBN over matrix Lie groups, their results were confined within a specific distance. In contrast, our work provides a more extensive examination, encompassing both population and sample properties of our LieBN in a general manner. Besides, all the discussion about our LieBN can be readily transferred to right-invariant metrics. This paper focuses on LieBN based on left-invariant metrics. Remark In contrast, our contribution lies in presenting a universally applicable result for all Lie groups with left-invariant metrics. In contrast, our contribution lies in presenting a universally applicable result for all Lie groups with left-invariant metrics. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv [3] When M corresponds to R equipped with the standard Euclidean metric, Eq. (12) reduces to the Euclidean Gaussian distribution Authors: (1) Ziheng Chen, University of Trento; (2) Yue Song, University of Trento and a Corresponding author; (3) Yunmei Liu, University of Louisville; (4) Nicu Sebe, University of Trento. Authors: Authors: (1) Ziheng Chen, University of Trento; (2) Yue Song, University of Trento and a Corresponding author; (3) Yunmei Liu, University of Louisville; (4) Nicu Sebe, University of Trento.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Abstract

Matrix

Riemannian Normalization on Lie Groups

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Brief Review of the Lie Group and the Geometries of SPD Manifolds

A Lie Group Approach to Riemannian Batch Normalization

Revisiting Euclidean Normalization: A Second Look

Revisiting Existing RBN: Developing Riemannian Normalization Approaches Tailored for Manifolds

LieBN on SPD Manifolds: Showcasing Our LieBN Framework

How We Implemented Our Three Families of LieBN to SPD Neural Networks

A Brief Review of the Lie Group and the Geometries of SPD Manifolds

A Lie Group Approach to Riemannian Batch Normalization

Revisiting Euclidean Normalization: A Second Look

Revisiting Existing RBN: Developing Riemannian Normalization Approaches Tailored for Manifolds

LieBN on SPD Manifolds: Showcasing Our LieBN Framework

How We Implemented Our Three Families of LieBN to SPD Neural Networks

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps