Authors:
(1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea;
(2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea;
(5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Authors: Authors: (1) Hyosun park, Department of Astronomy, Yonsei University, Seoul, Republic of Korea; (2) Yongsik Jo, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (3) Seokun Kang, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (4) Taehwan Kim, Artificial Intelligence Graduate School, UNIST, Ulsan, Republic of Korea; (5) M. James Jee, Department of Astronomy, Yonsei University, Seoul, Republic of Korea and Department of Physics and Astronomy, University of California, Davis, CA, USA. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Method 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.1. Overview and 2.2. Encoder-Decoder Architecture 2.3. Transformers for Image Restoration 2.3. Transformers for Image Restoration 2.4. Implementation Details 2.4. Implementation Details 3 Data and 3.1. HST Dataset 3 Data and 3.1. HST Dataset 3.2. GalSim Dataset 3.2. GalSim Dataset 3.3. JWST Dataset 3.3. JWST Dataset 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4 JWST Test Dataset Results and 4.1. PSNR and SSIM 4.2. Visual Inspection 4.2. Visual Inspection 4.3. Restoration of Morphological Parameters 4.3. Restoration of Morphological Parameters 4.4. Restoration of Photometric Parameters 4.4. Restoration of Photometric Parameters 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5 Application to real HST Images and 5.1. Restoration of Single-epoch Images and Comparison with Multi-epoch Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 5.2. Restoration of Multi-epoch HST Images and Comparison with Multi-epoch JWST Images 6 Limitations 6.1. Degradation in Restoration Quality Due to High Noise Level 6.1. Degradation in Restoration Quality Due to High Noise Level 6.2. Point Source Recovery Test 6.2. Point Source Recovery Test 6.3. Artifacts Due to Pixel Correlation 6.3. Artifacts Due to Pixel Correlation 7 Conclusions and Acknowledgements 7 Conclusions and Acknowledgements Appendix: A. Image restoration test with Blank Noise-Only Images Appendix: A. Image restoration test with Blank Noise-Only Images References References 6. LIMITATIONS 6.1. Degradation in Restoration Quality Due to High Noise Level Although our Transformer-based deep learning model provides state-of-the-art performance in both resolution enhancement and noise reduction for moderate noise levels, inevitably, proper restoration becomes impossible when the noise level exceeds a certain threshold. Figure 11 shows four such examples. The top row illustrates a case, where the distinct spiral arm in the GT image is not restored. Upon scrutiny of the corresponding LQ image, we believe that the information loss due to noise is too significant to hint at the presence of the spiral arm. The second row shows a case, where the position angle of the GT image is not properly restored. Again, we suspect that the LQ image is too noisy to enable a proper inference of the position angle of the GT image. The third row is an example, where the ellipticity of the RS image is much rounder than that of the GT image. In the last row, we illustrate a case, where the deep learning model fails to unveil the two distinct peaks of the GT image. Since the exact definition of restoration failure is a subjective matter, it is difficult to quote an exact threshold. Nevertheless, from visual inspections, we suggest that the failure frequency increases noticeably when the rms value (after min-max normalization) of the LQ image is larger than ∼0.1. 6.2. Point Source Recovery Test In principle, the perfect deconvolution algorithm should restore a point source to a delta function, which is infinitely smaller than a pixel. In traditional deconvolution in the Fourier domain, it is a challenging task because the operation is numerically unstable. The resulting images often exhibit many ringing effects around a bright central peak. Since we excluded stars from the training dataset, our deep learning model did not explicitly learn to deconvolve point source images. Thus, it is interesting to examine how well our deep learning model, trained with only galaxy images, restore point sources. We perform a point source recovery test as follows. First, we created 1,000 JWST-quality star GT images. Because we did not remove the JWST PSF from the training dataset, the star GT images should not resemble a delta functions but the JWST PSF. We implemented this by convolving a single pixel with a Gaussian whose kernel size matches the JWST PSF. Note that we randomize the positions of the stars within the central 24×24 pixels of the 64 × 64 postage-stamp images. Then, we created their LQ versions by further convolving the GT images with the HST PSF and adding noise. Finally, these LQ images are restored by our deep learning model. To investigate the systematic effect, we stack the 1,000 GT and RS images separately after aligning their centers. Figure 12 shows that the difference is subtle when we compare the GT (left) and RS (middle) stacks visually. However, the residual image (right) illustrates that the PSF of the RS stack is systematically larger. Thus, we conclude that our deep learning model, trained solely with galaxy images, performs less than ideally for point sources. 6.3. Artifacts Due to Pixel Correlation In our generation of LQ images, we assume that the noise is Gaussian. However, in real astronomical images, especially when we create deep images by stacking many dithered exposures, there exist significant interpixel noise correlations. We find that these inter-pixel noise correlations create non-negligible artifacts. Figure 13 display some examples of these artifacts. The LQ images here are sampled from multi-epoch drizzled images. The presence of correlated noise is apparent even from visual inspection. The RS images show that the correlated noise creates some low-surface brightness artifacts in the galaxy outskirts, which however are absent in the JWST images. Addressing this issue could involve strategies such as employing a different drizzling kernel for image stacking or leveraging more advanced deep learning algorithms. Exploring these solutions will be a key focus of our future work. This paper is available on arxiv under CC BY 4.0 Deed license. This paper is available on arxiv under CC BY 4.0 Deed license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Transformer-Based Restoration: Quantitative Gains and Boundaries in Space Data

AI Brings HST Images Closer to JWST Quality with Restormer

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

Even AI Needs Glasses: When Space Images Get Too Fuzzy to Fix

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

AI Breakthrough Sharpens Telescope Images-Astronomy’s Next Big Leap

Generative AI and Contextual Confidence: Abstract & Intro

Generative AI and Contextual Confidence: Challenges to Contextual Confidence from Generative AI

Generative AI and Contextual Confidence: Strategies to Promote Contextual Confidence

Generative AI and Contextual Confidence: Discussion, Acknowledgements and References

AI Breakthrough Sharpens Telescope Images-Astronomy’s Next Big Leap

Generative AI and Contextual Confidence: Abstract & Intro

Generative AI and Contextual Confidence: Challenges to Contextual Confidence from Generative AI

Generative AI and Contextual Confidence: Strategies to Promote Contextual Confidence

Generative AI and Contextual Confidence: Discussion, Acknowledgements and References

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps