Deep Neural Network for Sea Surface Temperature Prediction: Proposed Method

Table of Links

• Abstract and Intro
• Background
• Proposed Method
• Experimental Results and Analysis
• Conclusions and Future Work
• References

III. PROPOSED METHOD

Numerical model can predict the spatial distribution of SST and its global teleconnections together. It performs well at short-leads for SST prediction. Nevertheless, we argue that transfer the physical knowledge from the observed data can further improve the performance of numerical model for SST prediction. To this end, we adopt GANs to learn the physical knowledge in the observed data.

Zhu et al. [53] proposed a GAN inversion method that not only faithfully reconstructs the input data, but also ensures that the inverted latent code is semantically meaningful. They demonstrated that learning the pixel values of the target image alone is insufficient, and that the learned features are unable to represent the image at the semantic level. Inspired by this work, we design an encoder in GAN to learn physical knowledge from the observed data, referred to as the prior network. This prior network not only learns the pixel values of the target observed data, but also captures the physical information. It effectively improves the SST prediction accuracy.

Next, we present the proposed method as follows: 1) Overview of the method, 2) Prior network, 3) SST prediction with enhanced data.

A. Overview of the Method

In this subsection, we summarize the proposed SST prediction method and describe the input and output of each stage in detail. As illustrated in Fig. 2, the proposed SST prediction method consists of two stages: Prior network training and SST prediction with enhanced data.

1) Prior network training. This stage consists of three steps. In the first step, the observed SST (GHRST data) is used for GAN model training. In the second step, the pretrained generator and GHRSST data are used to train the encoder. In the third step, the pretrained generator and encoder are combined into the prior network. The prior network is used to transfer the physical knowledge from the observed data to the numerical model. The numerical model SST (HYCOM data) is then fed into the prior network to enhance its feature representations.

2) SST prediction with enhanced data. The physics-enhanced data are fed into ConvLSTM model for SST prediction. The SST of the next day, next 3 days, and next 7 days are predicted separately.

It should be noted that most existing works [26] [27] only use the observed data for ConvLSTM training. By contrast, our method takes advantage of physics-enhanced data for ConvLSTM training. Next, we describe prior network training and SST prediction with enhanced data in details.

B. Stage 1: Prior Network Training

We construct a prior network to learn the physical knowledge in the observed data and keep its semantic/physical information constant after training. As illustrated in Fig. 2, the prior network training is comprised of three steps: GAN model training, encoder training, and physics-enhanced data generation. Next we provide detailed descriptions of each step.

GAN Model Training. The GAN model is used to learning the data distribution from the observed SST. The objective function is as follows:

The training process of the GAN model is summarized in Algorithm 1. We train the model over the observed SST until the generator G captures the physical features from the observed SST data.

where F(·) represents feature extraction via the VGG network. VGG network stands for the network proposed by Visual Geometry Group [54], and it is a classical deep convolutional neural network.

The encoder training is described in Algorithm 2. The parameters of the generator G are fixed, while the parameters of encoder E and discriminator D are updated based on Eq. 2 and Eq. 3, respectively.

The motivation of Stage 1 is to construct a prior network which can rectify the incorrect components in the numerical model data. To this end, we firstly devise a GAN model which captures the data distribution from the observed SST and can generate high-quality SST data. Subsequently, the encoder is trained to guarantee that the generated latent codes preserve the semantic/physical information in the observed SST. We argue that through adversarial learning, the prior network (consisting of the encoder and generator) can rectify the incorrect parts in the input data, since the physical knowledge has been embedded in the prior network. Consequently, in the third step, when the numerical model data are fed into the prior network, the embedded physical knowledge can correct the incorrect components in the numerical model data.

C. Stage 2: SST Prediction with Enhanced Data

ConvLSTM is an effective tool for predicting spatialtemporal data. It is a recurrent neural network that incorporates convolutional blocks in both the input-to-state and state-tostate transitions. Unlike the traditional LSTM layer, ConvLSTM not only preserves the sequential relationship but also extracts spatial features from the data. In this way, we can leverage it to capture robust spatial-temporal features. The objective function of ConvLSTM is formulated as follows:

The physics-enhanced SST data are fed into the ConvLSTM model for SST prediction as follows:

The weights obtained by the generator are reused in Algorithm 2, where only the generator weights are fixed. The introduced encoder and the discriminator go through another training process over the observed SST. Their weights are updated based on Eq. 2 and Eq. 3, respectively. After training, the code generated by the encoder would embody the learned physical knowledge.

Finally, we acquire the data reinforced based on physical knowledge using the above pre-trained model. The weights of

the generator and the encoder from Algorithm 2 are reused and the numerical model SST is exploited to produce physics-reinforced numerical model data.

In Algorithm 3, the physical knowledge-enhanced data are leveraged to train a spatial-temporal ConvLSTM model for SST prediction. In this paper, the SST of the next day, the next 3 days and the next 7 days are predicted separately. For this part, we conducted an ablation study to make use of the reinforced data effectively.