Physics-Informed Machine Learning: Leveraging Physical Laws and Energy-Based Models

Table of Links

Abstract and 1. Introduction

2. Some recent trends in theoretical ML

   2.1 Deep Learning via continuous-time controlled dynamical system

   2.2 Probabilistic modeling and inference in DL

   2.3 Deep Learning in non-Euclidean spaces

   2.4 Physics Informed ML

3. Kuramoto model

   3.1 Kuramoto models from the geometric point of view

   3.2 Hyperbolic geometry of Kuramoto ensembles

   3.3 Kuramoto models with several globally coupled sub-ensembles

4. Kuramoto models on higher-dimensional manifolds

   4.1 Non-Abelian Kuramoto models on Lie groups

   4.2 Kuramoto models on spheres

   4.3 Kuramoto models on spheres with several globally coupled sub-ensembles

   4.4 Kuramoto models as gradient flows

   4.5 Consensus algorithms on other manifolds

5. Directional statistics and swarms on manifolds for probabilistic modeling and inference on Riemannian manifolds

   5.1 Statistical models over circles and tori

   5.2 Statistical models over spheres

   5.3 Statistical models over hyperbolic spaces

   5.4 Statistical models over orthogonal groups, Grassmannians, homogeneous spaces

6. Swarms on manifolds for DL

   6.1 Training swarms on manifolds for supervised ML

   6.2 Swarms on manifolds and directional statistics in RL

   6.3 Swarms on manifolds and directional statistics for unsupervised ML

   6.4 Statistical models for the latent space

   6.5 Kuramoto models for learning (coupled) actions of Lie groups

   6.6 Grassmannian shallow and deep learning

   6.7 Ensembles of coupled oscillators in ML: Beyond Kuramoto models

7. Examples

   7.1 Wahba’s problem

   7.2 Linked robot’s arm (planar rotations)

   7.3 Linked robot’s arm (spatial rotations)

   7.4 Embedding multilayer complex networks (Learning coupled actions of Lorentz groups)

8. Conclusion and References

2.4 Physics Informed ML

The term physics informed ML refers to the general approach aiming at exploiting physical knowledge in order to set up adequate models given the particular data set and the problem. In many cases, models and architectures are, at least partially, enforced by physical laws, such as symmetries or conservation laws [66, 65]. Taking this into account dramatically increases efficiency, transparency and robustness of ML algorithms. The very general idea standing behind this approach is the parsimony principle, one of the most universal principles in Science.

Although physics informed ML is regarded a very recent field, it has been developed upon the blend of ideas from computational physics and energy-based ML. Indeed, concepts of energy and entropy are built in early ML algorithms dealing with problems that are not necessarily related to any physical system [67]. A classical example of this kind is the famous Hopfield model. We also refer to [68] for energy-based approaches in RL.

More generally, the term theory informed ML refers to architectures which are imposed by a certain theoretical knowledge.

Approaches and models we propose in subsequent sections can be viewed as both physics informed and geometry informed ML. Moreover, many of them are also energy-based models.