Authors:
(1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea;
(2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Authors: Authors: (1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea; (2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Method 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.3. Transformers for Image Restoration 2.3. Transformers for Image Restoration 2.4. Implementation Details 2.4. Implementation Details 3 Data and 3.1. HST Dataset 3 Data and 3.1. HST Dataset 3.2. GalSim Dataset 3.2. GalSim Dataset 3.3. JWST Dataset 3.3. JWST Dataset 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4.2. Visual Inspection 4.2. Visual Inspection 4.3. Restoration of Morphological Parameters 4.3. Restoration of Morphological Parameters 4.4. Restoration of Photometric Parameters 4.4. Restoration of Photometric Parameters 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 6 Limitations 6.1. Degradation in Restoration Quality Due to High Noise Level 6.1. Degradation in Restoration Quality Due to High Noise Level 6.2. Point Source Recovery Test 6.2. Point Source Recovery Test 6.3. Artifacts Due to Pixel Correlation 6.3. Artifacts Due to Pixel Correlation 7 Conclusions and Acknowledgements 7 Conclusions and Acknowledgements Appendix: A. Image restoration test with Blank Noise-Only Images Appendix: A. Image restoration test with Blank Noise-Only Images References References 5. APPLICATION TO REAL HST IMAGES In §4, we have demonstrated that our deep learning model successfully restores the simulated HST images to the JWST-quality level in both resolution and depth. Here we present our results when the same model is applied to real HST images. It is important to note that we cannot perform identical comparisons as presented in §4 because there are no equivalent GT images for the real HST images. In §5.1, we restore single-epoch (short-exposure) images and compare their RS images to multi-epoch (long-exposure) images, using the latter as proxies for the GT dataset. Since our deep learn ing model not only reduces the noise level but also the effective PSF, the comparison cannot be fair because the multi-epoch image suffer from the same HST PSF. In addition, the multi-epoch image is not sufficiently deep and often noisier than the RS images. §5.1, we restore HST images and compare their RS images to deep JWST images. In this case, the resolutions between the two datasets are aligned. However, since their filters are different, we cannot expect the pairs to have identical photometric and morphological properties even if the deep learning model provides perfect restoration. 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images We utilize the datasets from the HUDF and GOODSN&S observed by the HST ACS. The single-epoch low-quality (SG-LQ) images are retrieved from the MAST archive. We aim to restore the FLT images not their “drizzled” (DRZ)images because “drizzling” induces inter-pixel noise correlations. When a postage-stamp image is contaminated by cosmic rays, we remove them with the lacosmic script[6]. Since the multi-epoch high quality (MT-HQ) mosaic images are corrected for geometric distortions unlike the FLT images, we transformed them back to the distorted, CCD coordinate system to enable a fair comparison with the FLT images. In Figure 8a, we display four restoration examples. Similar to the results observed in the experiment with the JWST test dataset, the restoration qualities are similarly excellent here. The low-surface brightness features, which cannot be identified in the LQ images, are well-restored in the RS images. One notable difference from the results in the JWST test dataset is the PSF size. Since our deep learning model is trained to restore the JWST resolution, the PSF size of the RS images are smaller than the HQ images. Another important difference is on average the higher noise level in our single-epoch LQ images. Figures 9 shows a comparison of the five morphological parameters. Significant improvements over the LQ images are clear in terms of the correlation strength and reduction of the scatter. Readers are reminded that we used the multi-epoch image as a proxy for the GT image after applying an inverse geometric distortion correction to it. Therefore, compared to the JWST dataset experiment, the measurement of the five morphological parameters from the GT surrogate is substantially noisier here. Also, as mentioned above, the PSF size of the RS image is smaller than that of the HQ image, which can contribute to a systematic difference in shapes between the two datasets. We display a comparison of photometric parameters in Figure 7. Again, significant improvements over the single-epoch LQ images are apparent. In particular, the pixel-to-pixel comparison shows a remarkable difference in both correlation strength and scatter reduction. Since the caveats mentioned in the morphological parameter comparison above apply here too, the improvements are less pronounced than those in the experiment with the JWST test dataset. 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images Figure 8b shows four examples. Despite the difference in filter, the RS images look remarkably similar to the JWST images. In particular, when the noise level is relatively low (i.e., top two panels), the restoration in both resolution and low-surface brightness feature is excellent. When the noise level is relatively high, the RS images still resemble the JWST images more closely than their input HST images. We suspect that the correlated noise in the input multi-epoch HST image might have non-negligible impact on the result in this noise regime. This paper is available on arxiv under CC BY 4.0 Deed license. This paper is available on arxiv under CC BY 4.0 Deed license. available on arxiv [6] https://lacosmic.readthedocs.io/en/stable

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Even AI Needs Glasses: When Space Images Get Too Fuzzy to Fix

Quantifying Restoration: PSNR, SSIM, and Morphology in Deep-Learned Galaxies

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

AI Brings HST Images Closer to JWST Quality with Restormer

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Transformer-Based Restoration: Quantitative Gains and Boundaries in Space Data

AI Breakthrough Sharpens Telescope Images-Astronomy’s Next Big Leap

Quantifying Restoration: PSNR, SSIM, and Morphology in Deep-Learned Galaxies

Even AI Needs Glasses: When Space Images Get Too Fuzzy to Fix

Transformer-Based Restoration: Quantitative Gains and Boundaries in Space Data

Transformer-Based Restoration: Quantitative Gains and Boundaries in Space Data

AI Breakthrough Sharpens Telescope Images-Astronomy’s Next Big Leap

Quantifying Restoration: PSNR, SSIM, and Morphology in Deep-Learned Galaxies

Even AI Needs Glasses: When Space Images Get Too Fuzzy to Fix

Transformer-Based Restoration: Quantitative Gains and Boundaries in Space Data

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps