Web Automation with Python and Seleniumby@terieyenike
1,728 reads

Web Automation with Python and Selenium

tldt arrow
EN
Read on Terminal Reader

Too Long; Didn't Read

Web automation is one of the best ways companies can test a product in development, especially the app's functionalities, such as clicking, scrolling, and other actions. In this article, we will learn how to use Selenium as an automation tool to test a website with Python and automate the entire process without using a mouse or keyboard on the browser. Selenium allows us to browse or use a browser without a human involved and automate processes such as typing into a user input and interacting with the website. For example, automating form submissions with Selenium is possible.
featured image - Web Automation with Python and Selenium
Teri Eyenike HackerNoon profile picture

@terieyenike

Teri Eyenike

Learn More
LEARN MORE ABOUT @TERIEYENIKE'S EXPERTISE AND PLACE ON THE INTERNET.
react to story with heart

Web automation is one of the best ways companies can test a product in development, especially the app's functionalities, such as clicking, scrolling, and other actions. 

Essentially, web automation is about mimicking human action as it is crucial to ensure the software works across all device types of users.

In this article, we will learn how to use Selenium as an automation tool to test a website with Python and automate the entire process without using a mouse or keyboard on the browser. Selenium doesn’t work with Python alone but with many other programming languages. 

What is Selenium Used For?

Selenium allows us to browse or use a browser without a human involved and automate processes through code, such as typing into a user input and interacting with the website.  For example, automating form submissions with Selenium is possible. Selenium does everything all by itself without a single click from a human.

Getting Started

Before Selenium can carry out any action through code, we need the packages and tools to be able to run browser automation and make it seamless. 

Installation

This section, we install the Selenium package and the WebDriver executable file. First, check the version of the Chrome browser so that it is equivalent to the same version of the WebDriver exec file.

image

The following needs to be installed on our local machine:

The Chrome browser. Download it here

Download the latest stable release of the ChromeDriver. On the ChromeDriver web page, check to confirm the version of Chrome matches the version of the ChromeDriver

Now, run this command to install Selenium in your terminal:

pip install selenium

Automating a Web Page

To begin, create a new folder containing the file and the downloaded ChromeDriver exec files. Selenium requires the driver to interface with the chosen browser, which in this tutorial, we are using the Chrome browser.

It is essential to place the ChromeDriver in the same directory as the Python file.

The directory for the project should look like this:

image

Another way to use the ChromeDriver in this project is to find the path location of the file and copy it, which should look like this:

MacBook users:

/Users/<computer user name>/Desktop/chromedriver

Windows users:

C:\Users\<computer user name\Desktop\chromdriver

The WebDriver is the bridge that bridges the Selenium code to work with the Chrome browser. The Chrome browser provides the bridge. Without it, automation will not be possible.

Next, let’s copy and paste the following code into the

automation.py
file:

# automation.py

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

chrome_driver_path = './chromedriver'

service = Service(chrome_drive_path)
driver = webdriver.Chrome(service=service)

url = "https://en.wikipedia.org/wiki/Main_Page"
driver.get(url)

article_count = driver.find_element(By.CSS_SELECTOR, "#articlecount a")
print(article_count.text)

drive.close()

The following occurs in the code snippet above as we will use Selenium to navigate Wikipedia:

  • Import the function webdriver from the selenium module
  • The
    selenium.webdriver
    module provides all the WebDriver implementations, and the Service object is for handling the browser driver
  • The
    By
    class is used to locate elements within the document of a web page using its CSS classes, ids, and so on
  • The Keys class simulates the actions from the keypress of the keyboard like the RETURN / Enter, F1, Alt, and so on
  • Assign the chromedriver exec file to a variable
  • Call the executable path of the driver to the service object
  • Next is to instantiate the Chrome browser with the
    webdriver.Chrome()
    with the service argument
  • Define the url of the web page to check
  • Using the
    .get()
    method will navigate to the page given by the url

On the Wikipedia page, open the inspect element to access the elements we will use to get the details from the page for automation.

Now to the task of finding the article count. WebDriver offers us a way to do so by targeting the element using the find_element method and using the

By.CSS_SELECTOR
as the first parameter with the selector name,
#articlecount a.

image


image

Before we run the Python program, the script has the method,

driver.close()
, that closes the browser window after running the test.

Run the program with the command:

python automation.py

The result from the terminal should display the article count, which shows the exact figure on the web page, 6,545,457.

Simulating a Click Action

From what we have done so far, it is evident that Selenium navigates a website without the use or click of a mouse.

Now, let’s simulate an action where the browser reacts to a click of a link on the page.

Update the

automation.py
with these lines of code:

# automation.py

# import modules

contact_us = driver.find_element(By.LINK_TEXT, "Contact us")

contact_us.click()

With this code snippet, we are finding the element to the link text, “Contact us” with the

By.LINK_TEXT
. After locating the elements, attach the
contact_us
to an action with
.click()
method.

Again, we run the command:

python automation.py


The command above gives the result of the contact us page.

Search a Web Page

There are so many use cases with Selenium. Searching a web page can be achieved with automation, which gives you the result for a search word.

Still, in the same file, automation.py, update the file with this code:

# automation.py

search = driver.find_element(By.NAME, "search")
search.send_keys("React", Keys.ENTER)

The code above does a similar thing to the previous by using the

find_element
method with the input's name attribute, search.

Also, the

send_keys
simulate pressing the ENTER key, searching the keyword React.

The result of the page should look like this:

image

Conclusion

This article is an overview of how to use Selenium in Python to automate the web, especially for professional developers who want to know if there are errors in the code and how to fix them.

The possibilities of using Selenium with Python for web automation are endless. Some use cases include filling out an online form, applying for LinkedIn jobs, playing a clicker game, and so on. Do you know other use cases for using Selenium? 

Kindly share your favorite use case in the comments below.

Learn More

Teri Eyenike HackerNoon profile picture
by Teri Eyenike @terieyenike.I am a software developer focused on creating content through technical writing and documentation.
Portfolio

RELATED STORIES

L O A D I N G
. . . comments & more!
Hackernoon hq - po box 2206, edwards, colorado 81632, usa