The Abstraction and Reasoning Corpus: Attention Masks for Core Geometry Priors

This paper is available on arxiv under CC 4.0 license. Authors: (1) Mattia Atzeni, EPFL, Switzerland and mattia.atzeni@outlook.it; (2) Mrinmaya Sachan, ETH Zurich, Switzerland; (3) Andreas Loukas, Prescient Design, Switzerland. Table of Links Abstract & Introduction Formalizing the Group-Action Learning Problem Attention Masks for Core Geometry Priors The LATFORMER Architecture Experiments Related Work Conclusion & References A. Additional Details on the Model B. Additional Experiments and Details on the Experimental Setup C. Limitations and Future Work D. Deferred Proofs 3. Attention Masks for Core Geometry Priors This section prepares some theoretical grounding for LATFORMER, our approach to learn the transformations for lattice symmetry groups in the form of attention masks. The section defines attention masks and explains how they can be leveraged to incorporate geometry priors when solving group action learning problems on sequences and images. 3.1. Modulating Attention Weights with Soft Masking 3.2. Existence of Attention Masks Implementing Lattice Symmetry Actions 3.3. Representing Attention Masks for Lattice Transformation To facilitate the learning of lattice symmetries, one needs to determine methods to parameterize the set of feasible group elements. Fortunately, as precised in the following theorem, the attention masks considered in Theorem 3.1 can be expressed conveniently under the same general formulation. Although strictly not a symmetry operation, scaling transformations of the lattice can also be defined in terms of attention masks under the same general formulation of Theorem 3.2, as reported in Table 1. Therefore, for completeness, we will consider scaling transformations as well in our experiments. Notice that Theorem 3.2 allows us to derive a way to calculate the attention masks. In particular, we can express our attention masks as a convolution operation on the identity, as stated below. This paper is available on arxiv under CC 4.0 license. Authors: (1) Mattia Atzeni, EPFL, Switzerland and mattia.atzeni@outlook.it; (2) Mrinmaya Sachan, ETH Zurich, Switzerland; (3) Andreas Loukas, Prescient Design, Switzerland. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Mattia Atzeni, EPFL, Switzerland and mattia.atzeni@outlook.it; (2) Mrinmaya Sachan, ETH Zurich, Switzerland; (3) Andreas Loukas, Prescient Design, Switzerland. Table of Links Abstract & Introduction Formalizing the Group-Action Learning Problem Attention Masks for Core Geometry Priors The LATFORMER Architecture Experiments Related Work Conclusion & References A. Additional Details on the Model B. Additional Experiments and Details on the Experimental Setup C. Limitations and Future Work D. Deferred Proofs Abstract & Introduction Abstract & Introduction Formalizing the Group-Action Learning Problem Formalizing the Group-Action Learning Problem Attention Masks for Core Geometry Priors Attention Masks for Core Geometry Priors The LATFORMER Architecture The LATFORMER Architecture Experiments Experiments Related Work Related Work Conclusion & References Conclusion & References A. Additional Details on the Model A. Additional Details on the Model B. Additional Experiments and Details on the Experimental Setup B. Additional Experiments and Details on the Experimental Setup C. Limitations and Future Work C. Limitations and Future Work D. Deferred Proofs D. Deferred Proofs 3. Attention Masks for Core Geometry Priors This section prepares some theoretical grounding for LATFORMER, our approach to learn the transformations for lattice symmetry groups in the form of attention masks. The section defines attention masks and explains how they can be leveraged to incorporate geometry priors when solving group action learning problems on sequences and images. 3.1. Modulating Attention Weights with Soft Masking 3.2. Existence of Attention Masks Implementing Lattice Symmetry Actions 3.3. Representing Attention Masks for Lattice Transformation To facilitate the learning of lattice symmetries, one needs to determine methods to parameterize the set of feasible group elements. Fortunately, as precised in the following theorem, the attention masks considered in Theorem 3.1 can be expressed conveniently under the same general formulation. Although strictly not a symmetry operation, scaling transformations of the lattice can also be defined in terms of attention masks under the same general formulation of Theorem 3.2, as reported in Table 1. Therefore, for completeness, we will consider scaling transformations as well in our experiments. Notice that Theorem 3.2 allows us to derive a way to calculate the attention masks. In particular, we can express our attention masks as a convolution operation on the identity, as stated below.