Authors:
(1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea;
(2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Authors: Authors: (1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea; (2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Method 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.3. Transformers for Image Restoration 2.3. Transformers for Image Restoration 2.4. Implementation Details 2.4. Implementation Details 3 Data and 3.1. HST Dataset 3 Data and 3.1. HST Dataset 3.2. GalSim Dataset 3.2. GalSim Dataset 3.3. JWST Dataset 3.3. JWST Dataset 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4.2. Visual Inspection 4.2. Visual Inspection 4.3. Restoration of Morphological Parameters 4.3. Restoration of Morphological Parameters 4.4. Restoration of Photometric Parameters 4.4. Restoration of Photometric Parameters 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 6 Limitations 6.1. Degradation in Restoration Quality Due to High Noise Level 6.1. Degradation in Restoration Quality Due to High Noise Level 6.2. Point Source Recovery Test 6.2. Point Source Recovery Test 6.3. Artifacts Due to Pixel Correlation 6.3. Artifacts Due to Pixel Correlation 7 Conclusions and Acknowledgements 7 Conclusions and Acknowledgements Appendix: A. Image restoration test with Blank Noise-Only Images Appendix: A. Image restoration test with Blank Noise-Only Images References References 3.2. GalSim Dataset For the generation of the pre-training dataset, we use GalSim based on the distributions of the Sersic parameters derived from the HST dataset to reproduce not only the marginalized distribution of each parameter but also the correlations between the parameters. We constructed the probability density functions (PDFs) in the five dimensional parameter space (axis ratio, Sersic index, flux, size, and noise) with the scipy.stats Gaussian kernel density estimator (KDE) module. Then, 100,000 new input parameter sets for GalSim were generated by resampling based on the derived PDFs. The image size w varies in the original HST postagestamp dataset. However, the galaxy images in the training dataset need to have an equal size. We choose a size of 64×64 pixels since the choice provides a good balance between efficiency and image fidelity. A smaller image would truncate the profile too prematurely while a larger image would be dominated by background values. The 100,000 noiseless galaxy images output by GalSim constitute the GT images in the pre-training dataset. Subsequently, these GT images were convolved with a 7×7 pixel Gaussian kernel whose width matches the HST PSF. Finally, Gaussian noise was added to align the rms noise level with that of the HST dataset. These degraded images comprise the LQ images. Following the same procedure used for determining the best-fit Sersic parameters from the HST galaxy images, we measured the Sersic parameters from the LQ images. Figure 4 compares the Sersic parameter distributions between the HST and GalSim LQ images. One must understand that in principle the two distributions cannot align exactly because the input Sersic parameters used for the GalSim image generation are derived from the noisy HST images. Nevertheless, we find that overall the covariances and marginalized probability distributions are similar between the two datasets. Since the GalSim galaxy images are used to pre-train the model, which is subsequently finetuned by the JWST-based images, we do not think that the differences in detail matter. 3.3. JWST Dataset We applied nearly the same procedures used for creating the HST dataset to sample the JWST image. The only difference is the criteria for the noise level. Since the JWST images prior to degrading serve as GT, the noise level must be considerably lower. We accepted an image only if its rms noise after the min-max normalization became less than 0.02. One non-trivial issue is how to handle the difference in pixel scale between JWST and HST. NIRCAM’s native pixel scale is 0.03′′ (0.06′′) for the short (long) wavelength channel, which is different from the HST/ACS pixel scale 0.05′′. We considered resampling the JWST pixel scale to 0.05′′. However, the interpolation noise degraded the original quality non-negligibly. Therefore, we maintained the original scale. Therefore, our JWST galaxies are 40% larger or 20% smaller than the HST ones on average. However, as we demonstrate in §4, our HST image restoration is not significantly affected by this pixel scale inconsistency. Another tricky issue is the JWST’s intrinsic PSF. Although JWST’s PSF is on average much smaller than that of HST, its long wavelength channel PSF width can be similar to that of the HST/ACS PSF. One may consider deconvolving JWST image with its own PSF as an attempt to further enhance its resolution and thus better represent the truth. Perhaps this is achievable with an approach similar to the current method explored here. However, this is beyond the scope of this study. In the current investigation, we use JWST images without deconvolution. Thus, the effective PSF of the LQ images that we describe below includes the convolutions by both JWST’s and HST’s PSFs. We standardize the postage-stamp image size to 64 × 64 pixels. A smaller image is expanded by padding pixels to its four edges in such a way that both the noise and background levels match the original image. A larger image is trimmed to 64 ×64 pixels. We discarded the galaxy images whose profiles were prematurely truncated by the trimming procedure. We collected 113,485 postage-stamp galaxy images from the JWST NIRCam F115W, F200W, F277W, and F444W images from various fields publicly available in August 2023 the Mikulski Archive for Space Telescopes (MAST)[4]. The dataset serves as the GT component in our finetuning dataset. The LQ counterpart is generated by convolving the JWST GT galaxy images with the HST PSF and adding Gaussian noises. Although the pixel scale is different from the HST image, we scaled the HST PSF size, assuming the 0.05′′ pixel scale. This is because, eventually, we desire to restore the HST images whose pixel scale is 0.05′′. The applied noise distribution follows that of the HST dataset. The ratios among the training, validation, and test datasets are 8:1:1. This paper is available on arxiv under CC BY 4.0 Deed license. This paper is available on arxiv under CC BY 4.0 Deed license. available on arxiv [4] https://archive.stsci.edu/

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Bridging Simulations and Observations: GalSim & JWST Data Drive Deep Learning

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

AI Breakthrough Sharpens Telescope Images-Astronomy’s Next Big Leap

10 Incredible Things Neil deGrasse Tyson’s New Book Will Teach You About the Universe

Are We in the Third or the Fourth Industrial Revolution?

In the Mind of an Astrophysicist: Meet the Writer TheCap

The Study of Neutrinos to Understand the Cosmos

Unveiling the Power of PM and P3M Algorithms on GPUs

AI Breakthrough Sharpens Telescope Images-Astronomy’s Next Big Leap

10 Incredible Things Neil deGrasse Tyson’s New Book Will Teach You About the Universe

Are We in the Third or the Fourth Industrial Revolution?

In the Mind of an Astrophysicist: Meet the Writer TheCap

The Study of Neutrinos to Understand the Cosmos

Unveiling the Power of PM and P3M Algorithms on GPUs

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps