paint-brush
Physics-Informed with Power-Enhanced Residual Network: Residual Networkby@interpolation
147 reads

Physics-Informed with Power-Enhanced Residual Network: Residual Network

by The Interpolation Publication
The Interpolation Publication HackerNoon profile picture

The Interpolation Publication

@interpolation

#1 Publication focused exclusively on Interpolation, ie determining value from...

February 28th, 2024
Read on Terminal Reader
Read this story in a terminal
Print this story
Read this story w/o Javascript
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

Discover the power of Power-Enhancing Residual Networks for superior interpolation in 2D/3D domains with physics-informed solutions, also available on Arxiv.
featured image - Physics-Informed with Power-Enhanced Residual Network: Residual Network
1x
Read by Dr. One voice-avatar

Listen to this story

The Interpolation Publication HackerNoon profile picture
The Interpolation Publication

The Interpolation Publication

@interpolation

#1 Publication focused exclusively on Interpolation, ie determining value from the existing values in a given data set.

Learn More
LEARN MORE ABOUT @INTERPOLATION'S
EXPERTISE AND PLACE ON THE INTERNET.

This paper is available on arxiv under CC 4.0 license.

Authors:

(1) Amir Noorizadegan, Department of Civil Engineering, National Taiwan University;

(2) D.L. Young, Core Tech System Co. Ltd, Moldex3D, Department of Civil Engineering, National Taiwan University & dlyoung@ntu.edu.tw;

(3) Y.C. Hon, Department of Mathematics, City University of Hong Kong;

(4) C.S. Chen, Department of Civil Engineering, National Taiwan University & dchen@ntu.edu.tw.

Abstract & Introduction

Neural Networks

PINN for Solving Inverse Burgers’ Equation

Residual Network

Numerical Results

Results, Acknowledgments & References

4 Residual Network

image


In this study, we propose a power-enhanced variant of the ResNet that skips every other layer, denoted as the “Power-Enhanced SkipResNet.” The modification involves altering the recursive definition in (9) as follows:


image


Figure 2: Three neural network architectures: (a) plain neural network (Plain NN), (b) residual network (ResNet), (c) power-enhanced SkipResNet, and (d) Unraveled SQRSkipResNet (plot (c) with p = 2) where ⊙ denotes element-wise multiplication.

Figure 2: Three neural network architectures: (a) plain neural network (Plain NN), (b) residual network (ResNet), (c) power-enhanced SkipResNet, and (d) Unraveled SQRSkipResNet (plot (c) with p = 2) where ⊙ denotes element-wise multiplication.


For the purpose of comparison among Plain NN, ResNet, and SQR-SkipResNet (Figs. 2(a)-(c), respectively), we evaluate the output of the third hidden layer concerning the input y0 = X . The results for the plain neural network are as follows:


image


Figure 2(d) visually represents the “expression tree” for the case with p = 2, providing an insightful illustration of the data flow from input to output. The graph demonstrates the existence of multiple paths that the data can traverse. Each of these paths represents a distinct configuration, determining which residual modules are entered and which ones are skipped.


Our extensive numerical experiments support our approach, indicating that a power of 2 is effective for networks with fewer than 30 hidden layers. However, for deeper networks, a larger power can contribute to network stability. Nonetheless, deploying such deep networks does not substantially enhance accuracy and notably increases CPU time. In tasks like interpolation and solving PDEs, a power of 2 generally suffices, and going beyond may not justify the added complexity in terms of accuracy and efficiency

L O A D I N G
. . . comments & more!

About Author

The Interpolation Publication HackerNoon profile picture
The Interpolation Publication@interpolation
#1 Publication focused exclusively on Interpolation, ie determining value from the existing values in a given data set.

TOPICS

THIS ARTICLE WAS FEATURED IN...

Permanent on Arweave
Read on Terminal Reader
Read this story in a terminal
 Terminal
Read this story w/o Javascript
Read this story w/o Javascript
 Lite
Unsafe
X REMOVE AD