paint-brush
Use Jupyter to Restart the Script from the Point Where the Scrapper Terminatedby@databug
218 reads

Use Jupyter to Restart the Script from the Point Where the Scrapper Terminated

by Andrew2mMarch 2nd, 2020
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

Using Jupyter, you can restart the script from the point where the scrapper has terminated. The solution depends on the “running code” feature, where we could run blocks of code independently from each batch. Python, Selenium, Pandas, Beautiful Soup, and your good old friend time are the library you need for this project. But do beware of the continuous error if the website server is unstable like my target website. The rest of the code is running :-)

Company Mentioned

Mention Thumbnail
featured image - Use Jupyter to Restart the Script from the Point Where the Scrapper Terminated
Andrew HackerNoon profile picture
Andrew

Andrew

@databug

Data Scientist who loves anything Data. From South Korea

About @databug
LEARN MORE ABOUT @DATABUG'S
EXPERTISE AND PLACE ON THE INTERNET.
L O A D I N G
. . . comments & more!

About Author

Andrew HackerNoon profile picture
Andrew@databug
Data Scientist who loves anything Data. From South Korea

TOPICS

THIS ARTICLE WAS FEATURED IN...

Permanent on Arweave
Read on Terminal Reader
Read this story in a terminal
 Terminal
Read this story w/o Javascript
Read this story w/o Javascript
 Lite
Coffee-web
Learnrepo
Its301
Pipfeed