Riemannian Normalization on Lie Groups

Table of Links

Abstract and 1 Introduction

2 Preliminaries

3. Revisiting Normalization

3.1 Revisiting Euclidean Normalization

3.2 Revisiting Existing RBN

4 Riemannian Normalization on Lie Groups

5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds

5.2 LieBN on SPD Manifolds

6 Experiments

6.1 Experimental Results

7 Conclusions, Acknowledgments, and References

APPENDIX CONTENTS

A Notations

B Basic layes in SPDnet and TSMNet

C Statistical Results of Scaling in the LieBN

D LieBN as a Natural Generalization of Euclidean BN

E Domain-specific Momentum LieBN for EEG Classification

F Backpropagation of Matrix Functions

G Additional Details and Experiments of LieBN on SPD manifolds

H Preliminary Experiments on Rotation Matrices

I Proofs of the Lemmas and Theories in the Main Paper

4 RIEMANNIAN NORMALIZATION ON LIE GROUPS

In this section, the neutral element in the Lie group M is denoted as E. Notably, the neutral element E is not necessarily the identity matrix. We first clarify the essential properties of Euclidean BN and then present our normalization method, tailored for Lie groups.

Two key points regarding Euclidean BN, as expressed by Eq. (6), are worth highlighting: (a) The standard BN (Ioffe & Szegedy, 2015) implicitly assumes a Gaussian distribution and can effectively normalize and transform the latent Gaussian distribution; (b) The centering and biasing operations control the mean, while the scaling controls the variance. Therefore, extending BN into Lie groups requires defining Gaussian distribution, centering, biasing, and scaling on Lie groups.

Remark 4.3. The MLE of the mean of the Gaussian distribution in Eq. (12) have been examined in several previous works (Said et al., 2017; Chakraborty & Vemuri, 2019; Chakraborty, 2020). However, these studies primarily focus on particular manifolds or specific metrics. In contrast, our contribution lies in presenting a universally applicable result for all Lie groups with left-invariant metrics. While Eq. (12) briefly appeared in Kobler et al. (2022b), the authors only focus on SPD manifolds under AIM. The transformation of the population under their proposed RBN remains unexplored as well. Besides, while Chakraborty (2020) analyzed the population properties for their RBN over matrix Lie groups, their results were confined within a specific distance. In contrast, our work provides a more extensive examination, encompassing both population and sample properties of our LieBN in a general manner. Besides, all the discussion about our LieBN can be readily transferred to right-invariant metrics. This paper focuses on LieBN based on left-invariant metrics.

[3] When M corresponds to R equipped with the standard Euclidean metric, Eq. (12) reduces to the Euclidean Gaussian distribution