Authors:
(1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea;
(2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Authors: Authors: (1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea; (2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Method 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.3. Transformers for Image Restoration 2.3. Transformers for Image Restoration 2.4. Implementation Details 2.4. Implementation Details 3 Data and 3.1. HST Dataset 3 Data and 3.1. HST Dataset 3.2. GalSim Dataset 3.2. GalSim Dataset 3.3. JWST Dataset 3.3. JWST Dataset 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4.2. Visual Inspection 4.2. Visual Inspection 4.3. Restoration of Morphological Parameters 4.3. Restoration of Morphological Parameters 4.4. Restoration of Photometric Parameters 4.4. Restoration of Photometric Parameters 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 6 Limitations 6.1. Degradation in Restoration Quality Due to High Noise Level 6.1. Degradation in Restoration Quality Due to High Noise Level 6.2. Point Source Recovery Test 6.2. Point Source Recovery Test 6.3. Artifacts Due to Pixel Correlation 6.3. Artifacts Due to Pixel Correlation 7 Conclusions and Acknowledgements 7 Conclusions and Acknowledgements Appendix: A. Image restoration test with Blank Noise-Only Images Appendix: A. Image restoration test with Blank Noise-Only Images References References 3. DATA The goal of the current study is to construct a Transformer model that enhances the quality of HST images to that of the JWST. Thus, our training data set consists of pairs of JWST-quality ground truth (GT) images and their degraded, low-quality (LQ) versions. We employ a transfer-learning approach by first pre-training our model with GT-LQ pairs of simplified galaxy images and finetuning the pre-trained model with GT-LQ pairs of realistic galaxy images. Therefore, we prepare two sets of training datasets. The GT image of the pre-training dataset is generated with GalSim (Rowe et al. 2015) while its LQ counterpart is obtained by simply degrading the resolution of the GT image with the HST point spread function (PSF) and adding Gaussian noise. The input parameters of GalSim should not be arbitrary but must be chosen carefully to represent the structural features of real galaxies. This is achieved by harvesting real galaxy images from the HST archive and measuring their properties. The GT image of the finetuning dataset is obtained by collecting galaxy images from the JWST archive. Following the same procedure in the pre-training dataset generation, we create their LQ counterparts. Below we describe the HST, GalSim, and JWST datasets in detail. 3.1. HST Dataset We collect multi-band galaxy images from the following HST/ACS deep fields: the Hubble Ultra Deep Field 12 (HUDF12), HUDF-Parallel 2 (HUDFP2)[2] (Ellis et al. 2013; Koekemoer et al. 2013), and the Great Observatories Origins Deep Survey North and South (GOODSN&S)[3] (Illingworth et al. 2016; Whitaker et al. 2019) fields. The F606W, F775W, and F814W filters are used from HUDF12. For HUDFP2, only F814W is selected. We employ F606W and F814W from GOODS-North&South. SExtractor (Bertin & Arnouts 1996) was used to detect sources by searching for five connected pixels 1.5 times above the rms noise level. We cropped a w × w pixel postage-stamp image centered on each source. The image size w is set to 2[4a+5], where a is the semi-major axis output by SExtractor and the operation [x] represents the greatest integer not exceeding x. We discard the source if its image size w is greater than 128 pixels,


either its semi-minor axis or half-light radius is smaller than the FWHM of HST/ACS PSF,


its S/N value is less than 20,


the maximum value of its neighboring sources is larger than 0.8 times its peak value, or


the gradient within the central 10 × 10 pixels is sharper than the HST PSF. its image size w is greater than 128 pixels, its image size w is greater than 128 pixels, either its semi-minor axis or half-light radius is smaller than the FWHM of HST/ACS PSF, either its semi-minor axis or half-light radius is smaller than the FWHM of HST/ACS PSF, its S/N value is less than 20, its S/N value is less than 20, the maximum value of its neighboring sources is larger than 0.8 times its peak value, or the maximum value of its neighboring sources is larger than 0.8 times its peak value, or the gradient within the central 10 × 10 pixels is sharper than the HST PSF. the gradient within the central 10 × 10 pixels is sharper than the HST PSF. The first and second criteria eliminate excessively large or small sources. The third condition removes excessively faint sources. The fourth condition excludes the cases where similarly bright galaxies are present within the postage stamp image. The last condition is necessary because the HST automatic cosmic ray detection/removal pipeline often fails near the bright peaks of stars or galaxies. In addition to these criteria, we explicitly identified and removed stars using their distinct locus in the size-magnitude diagram. We obtain a total of 66,092 postage-stamp images. Figure 3 shows the distribution of the HST postage-stamp images prior to the w = 128 cut. As mentioned above, our pre-training dataset is created by GalSim using the characteristics derived from our HST dataset. We choose to characterize HST galaxies by their best-fit elliptical Sersic parameters and fluxes. The circular Sersic profile is defined as where n represents the Sersic index, R is the radius, R50 is the half-light radius, and I(R50) is the surface brightness at R50. The elliptical Sersic profile is implemented by replacing R with the following: where a and b denote the semi-major and semi-minor axes, respectively. Because of the rapid change in I(R) near the center (R = 0), evaluating I(R) only at discrete positions (i.e., at the center of each pixel) leads to a significant discrepancy between the rendered and real galaxy images. To mitigate this, we first oversampled I(R) with ten times higher resolution in the central region (R < 5 pixel) and then applied 10 × 10 nearest neighbor-averaging to render the galaxy. This Sersic galaxy image is convolved with the HST PSF and then fit to the HST galaxy image. Fluxes are measured by summing up the backgroundsubtracted pixel values from the HST galaxy images. This paper is available on arxiv under CC BY 4.0 Deed license. This paper is available on arxiv under CC BY 4.0 Deed license. available on arxiv [2] https://archive.stsci.edu/prepds/hudf12/ [3] https://archive.stsci.edu/hlsps/hlf/

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Bridging Simulations and Observations: GalSim & JWST Data Drive Deep Learning

Efficient Transformers for Astronomical Images: Deconvolution and Denoising Unleashed

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

Space Selfies Level Up-AI Learns from the Best in the Universe

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Transformer-Based Restoration: Quantitative Gains and Boundaries in Space Data

It Is Okay If You Don't Know What You Like. We Do (feat. Deep Recommendation Algorithms)

10 Machine Learning, Data Science, and Deep Learning Courses for Programmers in 2020

10 Computer Vision Startups on Product Hunt with the Most Upvotes

10 Best Entry Level Machine Learning Tutorials

Transformer-Based Restoration: Quantitative Gains and Boundaries in Space Data

It Is Okay If You Don't Know What You Like. We Do (feat. Deep Recommendation Algorithms)

10 Machine Learning, Data Science, and Deep Learning Courses for Programmers in 2020

10 Computer Vision Startups on Product Hunt with the Most Upvotes

10 Best Entry Level Machine Learning Tutorials

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps