Adaptive Graph Neural Networks for Cosmological Data Generalization: Acknowledgements and Disclosure

Table of Links

Abstract and Intro

Data and Methods

Results

Conclusions

Acknowledgments and Disclosure of Funding, and References

Additional Plots

Acknowledgments and Disclosure of Funding

This manuscript has been supported by Fermi Research Alliance, LLC under Contract No. DE-AC02- 07CH11359 with the U.S. Department of Energy (DOE), Office of Science, Office of High Energy Physics.

This work was supported by the EU Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie Grant Agreement No. 690835, 734303, 822185, 858199, 101003460. The CAMELS project is supported by the Simons Foundation and the NSF grant AST2108078.

The authors of this paper have committed themselves to performing this work in an equitable, inclusive, and just environment, and we hold ourselves accountable, believing that the best science is contingent on a good research environment. We acknowledge the Deep Skies Lab as a community of multi-domain experts and collaborators who have facilitated an environment of open discussion, idea generation, and collaboration. This community was important for the development of this project.

Furthermore, we also thank the anonymous referees who helped improve this manuscript.

Author Contributions:

A. Roncoli: Methodology, Software, Validation, Formal analysis, Investigation, Writing - Original Draft, Visualization;

A. Ciprijanovi´c:´ Conceptualization, Methodology, Project administration, Resources, Software, Supervision, Writing - Original Draft, Funding Acquisition;

M. Voetberg: Software, Writing (review and editing);

F. Villaescusa-Navarro: Software, Writing (review and editing);

B. Nord: Supervision, Resources, Writing (review and editing).

References

[1] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.

[2] Stephon Alexander, Sergei Gleyzer, Hanna Parul, Pranath Reddy, Marcos Tidball, and Michael W. Toomey. Domain Adaptation for Simulation-based Dark Matter Searches with Strong Gravitational Lensing. Astrophysical Journal, 954(1):28, September 2023.

[3] Rene Andrae, Tim Schulze-Hartung, and Peter Melchior. Dos and don’ts of reduced chi-squared. arXiv preprint arXiv:1012.3754, 2010.

[4] Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018.

[5] Simeon Bird, Yueying Ni, Tiziana Di Matteo, Rupert Croft, Yu Feng, and Nianyi Chen. The ASTRID simulation: galaxy formation and reionization. Monthly Notices of the Royal Astronomical Society, 512(3):3703–3716, May 2022.

[6] Karsten M Borgwardt, Arthur Gretton, Malte J Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22(14):e49–e57, 2006.

[7] Ruichu Cai, Fengzhu Wu, Zijian Li, Pengfei Wei, Lingling Yi, and Kun Zhang. Graph domain adaptation: A generative view. arXiv preprint arXiv:2106.07482, 2021.

[8] A. Ciprijanovi ´ c, D. Kafkes, K. Downey, S. Jenkins, G. N. Perdue, S. Madireddy, T. Johnston, G. F. ´ Snyder, and B. Nord. DeepMerge - II. Building robust deep learning algorithms for merging galaxy identification across domains. Monthly Notices of the Royal Astronomical Society, 506(1):677–691, September 2021.

[9] A. Ciprijanovi ´ c, A. Lewis, K. Pedro, S. Madireddy, B. Nord, G. N. Perdue, and S. M. Wild. ´ DeepAstroUDA: semi-supervised universal domain adaptation for cross-survey galaxy morphology classification and anomaly detection. Machine Learning: Science and Technology, 4(2):025013, June 2023.

[10] Aleksandra Ciprijanovi ´ c, Diana Kafkes, Gregory Snyder, F. Javier Sánchez, Gabriel Nathan ´ Perdue, Kevin Pedro, Brian Nord, Sandeep Madireddy, and Stefan M. Wild. DeepAdveraries: examining the robustness of deep learning models for galaxy morphology classification. Machine Learning: Science and Technology, 3(3):035007, September 2022.

[11] Gabriela Csurka. Domain Adaptation for Visual Applications: A Comprehensive Survey. arXiv e-prints, page arXiv:1702.05374, February 2017.

[12] Quanyu Dai, Xiao-Ming Wu, Jiaren Xiao, Xiao Shen, and Dan Wang. Graph transfer learning via adversarial domain adaptation with graph convolution. IEEE Transactions on Knowledge and Data Engineering, 35(5):4908–4922, 2022.

[13] Romeel Davé, Daniel Anglés-Alcázar, Desika Narayanan, Qi Li, Mika H. Rafieferantsoa, and Sarah Appleby. SIMBA: Cosmological simulations with black hole growth and feedback. Monthly Notices of the Royal Astronomical Society, 486(2):2827–2849, June 2019.

[14] Natalí S. M. de Santi, Helen Shao, Francisco Villaescusa-Navarro, L. Raul Abramo, Romain Teyssier, Pablo Villanueva-Domingo, Yueying Ni, Daniel Anglés-Alcázar, Shy Genel, Elena Hernández-Martínez, Ulrich P. Steinwandel, Christopher C. Lovell, Klaus Dolag, Tiago Castro, and Mark Vogelsberger. Robust Field-level Likelihood-free Inference with Galaxies. Astrophysical Journal, 952(1):69, July 2023.

[15] DES and SPT Collaborations, T. M. C. Abbott, M. Aguena, A. Alarcon, O. Alves, A. Amon, F. Andrade-Oliveira, J. Annis, B. Ansarinejad, S. Avila, D. Bacon, and et al. Joint analysis of Dark Energy Survey Year 3 data and CMB lensing from SPT and Planck. III. Combined cosmological constraints. Phys. Rev. D, 107(2):023531, January 2023.

[16] DES Collaboration, T. M. C. Abbott, M. Aguena, A. Alarcon, S. Allam, O. Alves, A. Amon, F. Andrade-Oliveira, J. Annis, S. Avila, D. Bacon, and et. al. Dark Energy Survey Year 3 results: Cosmological constraints from galaxy clustering and weak lensing. Phys. Rev. D, 105(2):023520, January 2022.

[17] Zhengming Ding, Sheng Li, Ming Shao, and Yun Fu. Graph adaptive knowledge transfer for unsupervised domain adaptation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 37–52, 2018.

[18] Abolfazl Farahani, Sahar Voghoei, Khaled Rasheed, and Hamid R Arabnia. A brief review of domain adaptation. Advances in data science and information engineering: proceedings from ICDATA 2020 and IKE 2020, pages 877–894, 2021.

[19] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.

[20] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096–2030, 2016.

[21] Sankalp Gilda, Antoine de Mathelin, Sabine Bellstedt, and Guillaume Richard. Unsupervised Domain Adaptation for Constraining Star Formation Histories. arXiv e-prints, page arXiv:2112.14072, December 2021.

[22] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012.

[23] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.

[24] Lukas Hedegaard Morsing, Omar Ali Sheikh-Omar, and Alexandros Iosifidis. Supervised Domain Adaptation using Graph Embedding. arXiv e-prints, page arXiv:2003.04063, March 2020.

[25] Yesukhei Jagvaral, François Lanusse, Sukhdeep Singh, Rachel Mandelbaum, Siamak Ravanbakhsh, and Duncan Campbell. Galaxies and haloes on graph neural networks: Deep generative modelling scalar and vector quantities for intrinsic alignment. Monthly Notices of the Royal Astronomical Society, 516(2):2406–2419, October 2022.

[26] Niall Jeffrey and Benjamin D Wandelt. Solving high-dimensional parameter inference: marginal posterior densities & moment networks. arXiv preprint arXiv:2011.05991, 2020.

[27] Suruchi Kumari and Pravendra Singh. Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives. arXiv e-prints, page arXiv:2308.01265, July 2023.

[28] T. Lucas Makinen, Tom Charnock, Pablo Lemos, Natalia Porqueres, Alan F. Heavens, and Benjamin D. Wandelt. The Cosmic Graph: Optimal Information Extraction from Large-Scale Structure using Catalogues. The Open Journal of Astrophysics, 5(1):18, December 2022.

[29] Michelle Ntampaka, Daniel J Eisenstein, Sihan Yuan, and Lehman H Garrison. A hybrid deep learning approach to cosmological constraints from galaxy redshift surveys. The Astrophysical Journal, 889(2):151, 2020.

[30] Annalisa Pillepich, Volker Springel, Dylan Nelson, Shy Genel, Jill Naiman, Rüdiger Pakmor, Lars Hernquist, Paul Torrey, Mark Vogelsberger, Rainer Weinberger, and Federico Marinacci. Simulating galaxy formation with the IllustrisTNG model. Monthly Notices of the Royal Astronomical Society, 473(3):4077–4106, January 2018.

[31] Planck Collaboration, N. Aghanim, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi, M. Ballardini, A. J. Banday, R. B. Barreiro, N. Bartolo, S. Basak, and et al. Planck 2018 results. VI. Cosmological parameters. Astron. Astroph., 641:A6, September 2020.

[32] Dezso Ribli, Bálint Ármin Pataki, José Manuel Zorrilla Matilla, Daniel Hsu, Zoltán Haiman, and ˝ István Csabai. Weak lensing cosmology with convolutional neural networks on noisy data. Monthly Notices of the Royal Astronomical Society, 490(2):1843–1860, 2019.

[33] Helen Shao, Francisco Villaescusa-Navarro, Pablo Villanueva-Domingo, Romain Teyssier, Lehman H Garrison, Marco Gatti, Derek Inman, Yueying Ni, Ulrich P Steinwandel, Mihir Kulkarni, et al. Robust field-level inference of cosmological parameters with dark matter halos. The Astrophysical Journal, 944(1):27, 2023.

[34] Alex Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. A hilbert space embedding for distributions. In International conference on algorithmic learning theory, pages 13–31. Springer, 2007.

[35] Joshua B. Tenenbaum, Vin de Silva, and John C. Langford. A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science, 290(5500):2319–2323, December 2000.

[36] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7167–7176, 2017.

[37] Ricardo Vilalta, Kinjal Dhar Gupta, Dainis Boumber, and Mikhail M. Meskhi. A General Approach to Domain Adaptation with Applications in Astronomy. Publ. Astron. Soc. Pac., 131(1004):108008, October 2019.

[38] Francisco Villaescusa-Navarro, Daniel Anglés-Alcázar, Shy Genel, David N. Spergel, Rachel S. Somerville, Romeel Dave, Annalisa Pillepich, Lars Hernquist, Dylan Nelson, Paul Torrey, Desika Narayanan, Yin Li, Oliver Philcox, Valentina La Torre, Ana Maria Delgado, Shirley Ho, Sultan Hassan, Blakesley Burkhart, Digvijay Wadekar, Nicholas Battaglia, Gabriella Contardo, and Greg L. Bryan. The CAMELS Project: Cosmology and Astrophysics with Machine-learning Simulations. Astrophysical Journal, 915(1):71, July 2021.

[39] Francisco Villaescusa-Navarro, Shy Genel, Daniel Angles-Alcazar, David N Spergel, Yin Li, Benjamin Wandelt, Leander Thiele, Andrina Nicola, Jose Manuel Zorrilla Matilla, Helen Shao, et al. Robust marginalization of baryonic effects for cosmological inference at the field level. arXiv preprint arXiv:2109.10360, 2021.

[40] Francisco Villaescusa-Navarro, Shy Genel, Daniel Angles-Alcazar, Leander Thiele, Romeel Dave, Desika Narayanan, Andrina Nicola, Yin Li, Pablo Villanueva-Domingo, Benjamin Wandelt, et al. The camels multifield data set: Learning the universe’s fundamental parameters with artificial intelligence. The Astrophysical Journal Supplement Series, 259(2):61, 2022.

[41] Pablo Villanueva-Domingo and Francisco Villaescusa-Navarro. Learning Cosmology and Clustering with Cosmic Graphs. Astrophysical Journal, 937(2):115, October 2022.

[42] Pablo Villanueva-Domingo, Francisco Villaescusa-Navarro, Daniel Anglés-Alcázar, Shy Genel, Federico Marinacci, David N Spergel, Lars Hernquist, Mark Vogelsberger, Romeel Dave, and Desika Narayanan. Inferring halo masses with graph neural networks. The Astrophysical Journal, 935(1):30, 2022.

[43] Mei Wang and Weihong Deng. Deep Visual Domain Adaptation: A Survey. arXiv e-prints, page arXiv:1802.03601, February 2018.

[44] Man Wu, Shirui Pan, Chuan Zhou, Xiaojun Chang, and Xingquan Zhu. Unsupervised domain adaptive graph convolutional networks. In Proceedings of The Web Conference 2020, pages 1457–1467, 2020.

[45] Mengxi Wu and Mohammad Rostami. Unsupervised Domain Adaptation for Graph-Structured Data Using Class-Conditional Distribution Alignment. arXiv e-prints, page arXiv:2301.12361, January 2023.

[46] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4–24, 2020.

[47] Nan Yin, Li Shen, Mengzhu Wang, Long Lan, Zeyu Ma, Chong Chen, Xian-Sheng Hua, and Xiao Luo. Coco: A coupled contrastive framework for unsupervised domain adaptive graph classification. arXiv preprint arXiv:2306.04979, 2023.

[48] Wen Zhang and Dongrui Wu. Discriminative joint probability maximum mean discrepancy (djp-mmd) for domain adaptation. In 2020 international joint conference on neural networks (IJCNN), pages 1–8. IEEE, 2020.

[49] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI open, 1:57–81, 2020.