Bridging Geometry and Deep Learning: Key Developments in SPD and Grassmann Networks

Table of Links

Abstract and 1. Introduction

2. Preliminaries

3. Proposed Approach

   3.1 Notation

   3.2 Nueral Networks on SPD Manifolds

   3.3 MLR in Structure Spaces

   3.4 Neural Networks on Grassmann Manifolds

4. Experiments

5. Conclusion and References

A. Notations

B. MLR in Structure Spaces

C. Formulation of MLR from the Perspective of Distances to Hyperplanes

D. Human Action Recognition

E. Node Classification

F. Limitations of our work

G. Some Related Definitions

H. Computation of Canonical Representation

I. Proof of Proposition 3.2

J. Proof of Proposition 3.4

K. Proof of Proposition 3.5

L. Proof of Proposition 3.6

M. Proof of Proposition 3.11

N. Proof of Proposition 3.12

5 CONCLUSION

In this paper, we develop FC and convolutional layers for SPD neural networks, and MLR on SPSD manifolds. We show how to perform backpropagation with the Grassmann logarithmic map in the projector perspective. Based on this method, we extend GCNs to Grassmann geometry. Finally, we present our experimental results demonstrating the efficacy of our approach in the human action recognition and node classification tasks.

REFERENCES

Pierre-Antoine Absil, Robert E. Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2007.

Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Fast and Simple Computations on Tensors with Log-Euclidean Metrics. Technical Report RR-5584, INRIA, 2005.

E. Batzies, K. Huper, L. Machado, and F. Silva Leite. Geometric Mean and Geodesic Regression on ¨ Grassmannians. Linear Algebra and its Applications, 466:83–101, 2015.

Thomas Bendokat, Ralf Zimmermann, and P. A. Absil. A Grassmann Manifold Handbook: Basic Geometry and Computational Aspects. CoRR, abs/2011.13699, 2020. URL https://arxiv. org/abs/2011.13699.

Silvere Bonnabel, Anne Collard, and Rodolphe Sepulchre. Rank-preserving Geometric Means of ` Positive Semi-definite Matrices. Linear Algebra and its Applications, 438:3202–3216, 2013.

Daniel A. Brooks, Olivier Schwander, Fred´ eric Barbaresco, Jean-Yves Schneider, and Matthieu ´ Cord. Riemannian Batch Normalization for SPD Neural Networks. In NeurIPS, pp. 15463– 15474, 2019.

Rudrasis Chakraborty, Jose Bouza, Jonathan H. Manton, and Baba C. Vemuri. ManifoldNet: A Deep Neural Network for Manifold-valued Data with Applications. TPAMI, 44(2):799–810, 2020.

Ines Chami, Rex Ying, Christopher Re, and Jure Leskovec. Hyperbolic Graph Convolutional Neural ´ Networks. CoRR, abs/1910.12933, 2019. URL https://arxiv.org/abs/1910.12933.

Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Fully Hyperbolic Neural Networks. In ACL, pp. 5672–5686, 2022.

Calin Cruceru, Gary Becigneul, and Octavian-Eugen Ganea. Computationally Tractable Riemannian ´ Manifolds for Graph Embeddings. In AAAI, pp. 7133–7141, 2021.

Jindou Dai, Yuwei Wu, Zhi Gao, and Yunde Jia. A Hyperbolic-to-Hyperbolic Graph Convolutional Network. In CVPR, pp. 154–163, 2021.

Zhen Dong, Su Jia, Chi Zhang, Mingtao Pei, and Yuwei Wu. Deep Manifold Learning of Symmetric Positive Definite Matrices with Application to Face Recognition. In AAAI, pp. 4009–4015, 2017.

Alan Edelman, Tomas A. Arias, and Steven T. Smith. The Geometry of Algorithms with Orthogo- ´ nality Constraints. SIAM Journal on Matrix Analysis and Applications, 20(2):303–353, 1998.

Octavian-Eugen Ganea, Gary Becigneul, and Thomas Hofmann. Hyperbolic neural networks. In ´ NeurIPS, pp. 5350–5360, 2018.

Guillermo Garcia-Hernando, Shanxin Yuan, Seungryul Baek, and Tae-Kyun Kim. First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations. In CVPR, pp. 409–419, 2018.

William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive Representation Learning on Large Graphs. In NIPS, pp. 1025–1035, 2017.

Mehrtash Harandi, Mathieu Salzmann, and Richard Hartley. Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods. TPAMI, 40:48–62, 2018.

Zhiwu Huang and Luc Van Gool. A Riemannian Network for SPD Matrix Learning. In AAAI, pp. 2036–2042, 2017.

Zhiwu Huang, Jiqing Wu, and Luc Van Gool. Building Deep Networks on Grassmann Manifolds. In AAAI, pp. 3279–3286, 2018.

Catalin Ionescu, Orestis Vantzos, and Cristian Sminchisescu. Matrix Backpropagation for Deep Networks with Structured Layers. In ICCV, pp. 2965–2973, 2015.

Ce Ju and Cuntai Guan. Graph Neural Networks on SPD Manifolds for Motor Imagery Classification: A Perspective From the Time-Frequency Analysis. IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, 2023.

Qiyu Kang, Kai Zhao, Yang Song, Sijie Wang, and Wee Peng Tay. Node Embedding from Neural Hamiltonian Orbits in Graph Neural Networks. In ICML, pp. 15786–15808, 2023.

Sejong Kim. Ordered Gyrovector Spaces. Symmetry, 12(6), 2020.

Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. CoRR, abs/1609.02907, 2017. URL https://arxiv.org/abs/1609.02907.

Reinmar J. Kobler, Jun ichiro Hirayama, Qibin Zhao, and Motoaki Kawanabe. SPD Domainspecific Batch Normalization to Crack Interpretable Unsupervised Domain Adaptation in EEG. In NeurIPS, pp. 6219–6235, 2022.

Guy Lebanon and John Lafferty. Hyperplane Margin Classifiers on the Multinomial Manifold. In ICML, pp. 66, 2004.

Zhenhua Lin. Riemannian Geometry of Symmetric Positive Definite Matrices via Cholesky Decomposition. SIAM Journal on Matrix Analysis and Applications, 40(4):1353–1370, 2019.

Qi Liu, Maximilian Nickel, and Douwe Kiela. Hyperbolic Graph Neural Networks. In NeurIPS, pp. 8228–8239, 2019.

Ziyu Liu, Hongwen Zhang, Zhenghao Chen, Zhiyong Wang, and Wanli Ouyang. Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition. In CVPR, pp. 143–152, 2020.

Federico Lopez, Beatrice Pozzetti, Steve Trettel, Michael Strube, and Anna Wienhard. Vector- ´ valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices. In NeurIPS, pp. 18350–18366, 2021.

Meinard Muller, Tido R ¨ oder, Michael Clausen, Bernhard Eberhardt, Bj ¨ orn Kr ¨ uger, and Andreas ¨ Weber. Documentation Mocap Database HDM05. Technical Report CG-2007-2, Universitat¨ Bonn, June 2007.

Vinod Nair and Geoffrey E. Hinton. Rectified Linear Units Improve Restricted Boltzmann Machines. In ICML, pp. 807–814, 2010.

Galileo Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven Active Surveying for Collective Classification. In 10th International Workshop on Mining and Learning with Graphs, volume 8, pp. 1, 2012a.

Galileo Namata, Ben London, Lise Getoor, and Bert Huang. Query-driven Active Surveying for Collective Classification. In Workshop on Mining and Learning with Graphs, 2012b.

Xuan Son Nguyen. GeomNet: A Neural Network Based on Riemannian Geometries of SPD Matrix Space and Cholesky Space for 3D Skeleton-Based Interaction Recognition. In ICCV, pp. 13379– 13389, 2021.

Xuan Son Nguyen. A Gyrovector Space Approach for Symmetric Positive Semi-definite Matrix Learning. In ECCV, pp. 52–68, 2022a.

Xuan Son Nguyen. The Gyro-Structure of Some Matrix Manifolds. In NeurIPS, pp. 26618–26630, 2022b.

Xuan Son Nguyen and Shuo Yang. Building Neural Networks on Matrix Manifolds: A Gyrovector Space Approach. CoRR, abs/2305.04560, 2023. URL https://arxiv.org/abs/2305. 04560.

Xuan Son Nguyen, Luc Brun, Olivier Lezoray, and S ´ ebastien Bougleux. A Neural Network Based ´ on SPD Manifold Learning for Skeleton-based Hand Gesture Recognition. In CVPR, pp. 12036– 12045, 2019.

Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A Riemannian Framework for Tensor Computing. Technical Report RR-5255, INRIA, 2004.

Xavier Pennec, Stefan Horst Sommer, and Tom Fletcher. Riemannian Geometric Statistics in Medical Image Analysis. Academic Press, 2020.

Chiara Plizzari, Marco Cannici, and Matteo Matteucci. Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks. Computer Vision and Image Understanding, 208: 103219, 2021.

Prithviraj Sen, Galileo Mark Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Eliassi-Rad. Collective Classification in Network Data. AI Magazine, 29(3):93–106, 2008.

Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis. In CVPR, pp. 1010–1019, 2016.

Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic Neural Networks++. CoRR, abs/2006.08210, 2021. URL https://arxiv.org/abs/2006.08210.

Ondrej Skopek, Octavian-Eugen Ganea, and Gary Becigneul. Mixed-curvature Variational Autoencoders. CoRR, abs/1911.08411, 2020. URL https://arxiv.org/abs/1911.08411.

Lincon S. Souza, Naoya Sogi, Bernardo B. Gatto, Takumi Kobayashi, and Kazuhiro Fukui. An Interface between Grassmann Manifolds and Vector Spaces. In CVPRW, pp. 3695–3704, 2020.

Rhea Sanjay Sukthanker, Zhiwu Huang, Suryansh Kumar, Erik Goron Endsjo, Yan Wu, and Luc Van Gool. Neural Architecture Search of SPD Manifold Networks. In IJCAI, pp. 3002–3009, 2021.

Abraham Albert Ungar. Beyond the Einstein Addition Law and Its Gyroscopic Thomas Precession: The Theory of Gyrogroups and Gyrovector Spaces. Fundamental Theories of Physics, vol. 117, Springer, Netherlands, 2002.

Abraham Albert Ungar. Analytic Hyperbolic Geometry: Mathematical Foundations and Applications. World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2005.

Abraham Albert Ungar. Analytic Hyperbolic Geometry in N Dimensions: An Introduction. CRC Press, 2014.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li ´ o, and Yoshua ` Bengio. Graph Attention Networks. CoRR, abs/1710.10903, 2018. URL https://arxiv. org/abs/1710.10903.

Rui Wang and Xiao-Jun Wu. GrasNet: A Simple Grassmannian Network for Image Set Classification. Neural Processing Letters, 52(1):693–711, 2020.

Rui Wang, Xiao-Jun Wu, and Josef Kittler. SymNet: A Simple Symmetric Positive Definite Manifold Deep Learning Method for Image Set Classification. IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, 2021.

Muhan Zhang and Yixin Chen. Link Prediction Based on Graph Neural Networks. CoRR, abs/1802.09691, 2018. URL https://arxiv.org/abs/1802.09691.

Yiding Zhang, Xiao Wang, Chuan Shi, Nian Liu, and Guojie Song. Lorentzian Graph Convolutional Networks. In Proceedings of the Web Conference 2021, pp. 1249–1261, 2021.

Yiding Zhang, Xiao Wang, Chuan Shi, Xunqiang Jiang, and Yanfang Ye. Hyperbolic Graph Attention Network. IEEE Transactions on Big Data, 8(6):1690–1701, 2022.

Wei Zhao, Federico Lopez, J. Maxwell Riestenberg, Michael Strube, Diaaeldin Taha, and Steve Trettel. Modeling Graphs Beyond Hyperbolic: Graph Neural Networks in Symmetric Positive Definite Matrices. CoRR, abs/2306.14064, 2023. URL https://arxiv.org/abs/2306. 14064.

Bingxin Zhou, Xuebin Zheng, Yu Guang Wang, Ming Li, and Junbin Gao. Embedding Graphs on Grassmann Manifold. Neural Networks, 152:322–331, 2022.

Huanyu Zhou, Qingjie Liu, and Yunhong Wang. Learning Discriminative Representations for Skeleton Based Action Recognition. In CVPR, pp. 10608–10617, 2023.