Wahba’s Problem and SO(3) Optimization: Rotation Learning in Geometric ML

Table of Links

Abstract and 1. Introduction

2. Some recent trends in theoretical ML

   2.1 Deep Learning via continuous-time controlled dynamical system

   2.2 Probabilistic modeling and inference in DL

   2.3 Deep Learning in non-Euclidean spaces

   2.4 Physics Informed ML

3. Kuramoto model

   3.1 Kuramoto models from the geometric point of view

   3.2 Hyperbolic geometry of Kuramoto ensembles

   3.3 Kuramoto models with several globally coupled sub-ensembles

4. Kuramoto models on higher-dimensional manifolds

   4.1 Non-Abelian Kuramoto models on Lie groups

   4.2 Kuramoto models on spheres

   4.3 Kuramoto models on spheres with several globally coupled sub-ensembles

   4.4 Kuramoto models as gradient flows

   4.5 Consensus algorithms on other manifolds

5. Directional statistics and swarms on manifolds for probabilistic modeling and inference on Riemannian manifolds

   5.1 Statistical models over circles and tori

   5.2 Statistical models over spheres

   5.3 Statistical models over hyperbolic spaces

   5.4 Statistical models over orthogonal groups, Grassmannians, homogeneous spaces

6. Swarms on manifolds for DL

   6.1 Training swarms on manifolds for supervised ML

   6.2 Swarms on manifolds and directional statistics in RL

   6.3 Swarms on manifolds and directional statistics for unsupervised ML

   6.4 Statistical models for the latent space

   6.5 Kuramoto models for learning (coupled) actions of Lie groups

   6.6 Grassmannian shallow and deep learning

   6.7 Ensembles of coupled oscillators in ML: Beyond Kuramoto models

7. Examples

   7.1 Wahba’s problem

   7.2 Linked robot’s arm (planar rotations)

   7.3 Linked robot’s arm (spatial rotations)

   7.4 Embedding multilayer complex networks (Learning coupled actions of Lorentz groups)

8. Conclusion and References

7 Examples

We present several illustrative problems in order to support the main points of this article. We intentionally choose relatively simple examples in order to illustrate how particular problems can be addressed by appropriately chosen models from the proposed general framework.

For the sake of brevity of exposition we omit technical details on implementation. The details and results will be presented elsewhere.

7.1 Wahba’s problem

Wahba’s problem [141] can be regarded as a classical ML problem over Riemannian manifolds, although it has been stated in 1965 before the advent of ML. The problem consists in finding a rotation in the three-dimensional space (SO(3) matrix) from a set of noisy (possibly weighted) observations. It can be stated as follows.

Hence, this is an optimization problem over SO(3). It might be considered as a problem of "shallow" ML. There are efficient solutions which exploit the apparatus of linear algebra (for instance, SVD matrix decomposition).

Recent papers [59, 64] on RL on manifolds tackled Wahba’s problem in order to illustrate stochastic policies over SO(3). Both papers experimented with the Bingham policy parametrization.

The remaining two families exposed in subsection 5.2 (von Mises-Fisher and Bergman-Cauchy) can also be used for the policy parametrization over S3. We believe that Cauchy and Bergman-Cauchy are more convenient options than the von Mises-Fisher and Bingham because of their group-invariance properties.