Hello Pythonistas, In this tutorial you're are going to learn out how to track worldwide cases using requests and BeautifulSoup library in Python. Coronavirus : If you're new to I would recommend checking out firstly and then come back to complete this tutorial. Note webscraping A beginner guide to Webscraping Requirements To effectively follow through with this tutorial, you need to have the following libraries installed on your machine with exception of CSV Module which comes with Python standard library. requests BeautfulSoup CSV Installation Just use the pip to install the mentioned dependencies just as shown below; $ pip install requests

$ pip install beautifulsoup4 Let's get started where do scrap the corona cases? As we have stated above we are going to be the number of cases worldwide by scraping the information from the cloud, there many choices of websites to scrap from but in this tutorial, we will with . tracking worldometer Let's figure out the structure of the website Let's firstly understand the structure of the website we are about to scrap, once you open up the website, and then you will see a table similar to what shown below. worldometer scroll down How table is represented in HTML ? The table in HTML is usually represented with a tag whereby indicate and indicates a specific in that low just as shown in the example below; table tr row td column Row 1, Column 1 Row 1, Column 2 Row 2, Column 1 Row 2, Column 2 < = > table border "1" < > tr < > td </ > td < > td </ > td </ > tr < > tr < > td </ > td < > td </ > td </ > tr </ > table That means every row in the table is represented by tag , therefore we need to filter all rows from table then store in CSV file. worldometers tr the worldometers Complete Coronavirus Spider Below is the complete code of the spider that you're going to build in this tutorial which is capable of scraping the live Coronavirus number from website and then store them in a CSV file. worldometer app.py ​ csv requests bs4 BeautifulSoup

html = requests.get( ).text
html_soup = BeautifulSoup(html, )
rows = html_soup.find_all( ) element = BeautifulSoup(row, ).find_all(tag)
    text = [col.get_text() col element] text

heading = rows.pop( )
heading_row = extract_text(str(heading), )[ : ] open( , ) store:
    Store = csv.writer(store, delimiter= )
    Store.writerow(heading_row) row rows:
        test_data = extract_text(str(row), )[ : ]
        Store.writerow(test_data) import import from import 'https://www.worldometers.info/coronavirus/' 'html.parser' 'tr' : def extract_text (row, tag) 'html.parser' for in return 0 'th' 1 9 with 'corona.csv' 'w' as ',' for in 'td' 1 9 OUTPUT Let's break the code into pieces so as we can understand the role of each part and how they all work together in getting realtime updated numbers. Importing necessary libraries The first 3 lines of code just import all our necessary modules and libraries that we will be using to scrap the live covid19 cases and storing data in the file. csv requests bs4 BeautifulSou import import from import Getting the web-page In order for us to extract and filter coronavirus numbers, we need a way to programmatically access the source code of the webpage, in doing this we will use the requests library just as shown below. html = requests.get( ).text 'https://www.worldometers.info/coronavirus/' Extracting all rows in a table Now that we have the HTML source code of the website, it's now time to parse all the rows present in that table showing coronavirus stats, in doing this will use beautifulSoup just as shown below. html_soup = BeautifulSoup(html, )

rows = html_soup.find_all( ) 'html.parser' 'tr' Making a function do unpack the row columns After extracting all the rows in the table, we need a way to parse all the details on n that row, and that's what the below function does. coronavirus each column i element = BeautifulSoup(row, ).find_all(tag)
    text = [col.get_text() col element] text : def extract_text (row, tag) 'html.parser' for in return Parsing the header Since we don't wanna confuse the header naming and real stats, we need to out the header from the row list as shown below. pop heading = rows.pop( )

heading_row = extract_text(str(heading), )[ : ] 0 'th' 1 9 Parsing row details and storing to CSV Now finally our last job is to parse all the individual details of every row in the table and then store in the file using the module as shown below; CSV CSV Congratulations you have just made your own How to track Coronavirus in Python, to share it with your fellow developers. Tweet now Based on your interest, you might also love these articles; A beginner guide to web scraping How to track phone number in Python The basics of requests module in Python How to extract all website links in Python How to control your Arduino with python How to remove duplicates on your drive using python In case of any comment, suggestion, or difficulties drop it in the comment box below and I will get back to you ASAP. To get the full code for this article you can check out on My Github Previously published at https://kalebujordan.com/scrap-worldometers-live-update-with-beautifulsoup/

Read My Stories

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

How to Track Pandemic Cases With Python

Too Long; Didn't Read

Company Mentioned

Kalebu Jordan

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

How to Track Pandemic Cases With Python

Too Long; Didn't Read

Company Mentioned

Kalebu Jordan

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES