Table of Links Abstract and 1 Introduction 2 Preliminaries 3. Revisiting Normalization 3.1 Revisiting Euclidean Normalization 3.2 Revisiting Existing RBN 4 Riemannian Normalization on Lie Groups 5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds 5.2 LieBN on SPD Manifolds 6 Experiments 6.1 Experimental Results 7 Conclusions, Acknowledgments, and References APPENDIX CONTENTS A Notations B Basic layes in SPDnet and TSMNet C Statistical Results of Scaling in the LieBN D LieBN as a Natural Generalization of Euclidean BN E Domain-specific Momentum LieBN for EEG Classification F Backpropagation of Matrix Functions G Additional Details and Experiments of LieBN on SPD manifolds H Preliminary Experiments on Rotation Matrices I Proofs of the Lemmas and Theories in the Main Paper 3.1 REVISITING EUCLIDEAN NORMALIZATION In Euclidean DNNs, normalization stands as a pivotal technique for accelerating network training by mitigating the issue of internal covariate shift (Ioffe & Szegedy, 2015). While various normalization methods have been introduced (Ioffe & Szegedy, 2015; Ba et al., 2016; Ulyanov et al., 2016; Wu & He, 2018), they all share a common fundamental concept: the regulation of the first and second statistical moments. In this paper, we focus on batch normalization only. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors:
(1) Ziheng Chen, University of Trento;
(2) Yue Song, University of Trento and a Corresponding author;
(3) Yunmei Liu, University of Louisville;
(4) Nicu Sebe, University of Trento. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Preliminaries 2 Preliminaries 3. Revisiting Normalization 3.1 Revisiting Euclidean Normalization 3.1 Revisiting Euclidean Normalization 3.2 Revisiting Existing RBN 3.2 Revisiting Existing RBN 4 Riemannian Normalization on Lie Groups 4 Riemannian Normalization on Lie Groups 5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds 5 LieBN on the Lie Groups of SPD Manifolds and 5.1 Deformed Lie Groups of SPD Manifolds 5.2 LieBN on SPD Manifolds 5.2 LieBN on SPD Manifolds 6 Experiments 6 Experiments 6.1 Experimental Results 6.1 Experimental Results 7 Conclusions, Acknowledgments, and References 7 Conclusions, Acknowledgments, and References APPENDIX CONTENTS APPENDIX CONTENTS A Notations A Notations B Basic layes in SPDnet and TSMNet B Basic layes in SPDnet and TSMNet C Statistical Results of Scaling in the LieBN C Statistical Results of Scaling in the LieBN D LieBN as a Natural Generalization of Euclidean BN D LieBN as a Natural Generalization of Euclidean BN E Domain-specific Momentum LieBN for EEG Classification E Domain-specific Momentum LieBN for EEG Classification F Backpropagation of Matrix Functions F Backpropagation of Matrix Functions G Additional Details and Experiments of LieBN on SPD manifolds G Additional Details and Experiments of LieBN on SPD manifolds H Preliminary Experiments on Rotation Matrices H Preliminary Experiments on Rotation Matrices I Proofs of the Lemmas and Theories in the Main Paper I Proofs of the Lemmas and Theories in the Main Paper 3.1 REVISITING EUCLIDEAN NORMALIZATION In Euclidean DNNs, normalization stands as a pivotal technique for accelerating network training by mitigating the issue of internal covariate shift (Ioffe & Szegedy, 2015). While various normalization methods have been introduced (Ioffe & Szegedy, 2015; Ba et al., 2016; Ulyanov et al., 2016; Wu & He, 2018), they all share a common fundamental concept: the regulation of the first and second statistical moments. In this paper, we focus on batch normalization only. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv Authors: (1) Ziheng Chen, University of Trento; (2) Yue Song, University of Trento and a Corresponding author; (3) Yunmei Liu, University of Louisville; (4) Nicu Sebe, University of Trento. Authors: Authors: (1) Ziheng Chen, University of Trento; (2) Yue Song, University of Trento and a Corresponding author; (3) Yunmei Liu, University of Louisville; (4) Nicu Sebe, University of Trento.

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Abstract

Matrix

Revisiting Euclidean Normalization: A Second Look

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Brief Review of the Lie Group and the Geometries of SPD Manifolds

A Lie Group Approach to Riemannian Batch Normalization

Revisiting Existing RBN: Developing Riemannian Normalization Approaches Tailored for Manifolds

Riemannian Normalization on Lie Groups

LieBN on SPD Manifolds: Showcasing Our LieBN Framework

How We Implemented Our Three Families of LieBN to SPD Neural Networks

A Brief Review of the Lie Group and the Geometries of SPD Manifolds

A Lie Group Approach to Riemannian Batch Normalization

Revisiting Existing RBN: Developing Riemannian Normalization Approaches Tailored for Manifolds

Riemannian Normalization on Lie Groups

LieBN on SPD Manifolds: Showcasing Our LieBN Framework

How We Implemented Our Three Families of LieBN to SPD Neural Networks

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps