Greetings to all Cypress enthusiasts! In this article, we’ll explore one interesting way of running Cypress tests across multiple browsers in CI/CD using Docker. The cool thing here is that with the help of Docker Compose we will set up the simultaneous launch of several Docker containers in which Cypress tests will be executed in parallel in different browsers. The entire workflow will run automatically on the GitHub Actions platform, and as a result, we will get artifacts from running tests in each of the browsers. Get ready to read, it’s going to be very exciting! Why Cross Browser Testing? is one of the most important characteristics of a web application, which implies its equally correct display and functionality in various browsers and their versions. Cross Browser compatibility The modern variety of browsers is determined, among other things, by the ways of web application content, when different browser engines ( ) perceive and process HTML tags and CSS styles differently, which naturally affects the appearance and application behavior. differences in rendering Blink, WebKit, Gecko, EdgeHTML In this regard, the importance of is obvious and extremely clear. Its purpose is to ensure that when a user opens a web application in various browsers and their versions, the correct display of its content is ensured, the integrity of its structure is preserved, there are no functional errors and inconsistencies in performance, layout collapse, overlapping elements on top of each other, etc. cross-browser testing Cypress Features in Cross Browser Testing If you are not yet familiar with : Cypress Cypress is a JavaScript-based end-to-end testing tool designed for modern web test automation. Cypress allows conducting both full-fledged end-to-end testing with passing user scenarios on a real product, as well as integration testing of individual front-end components. Cypress has emerged as a popular end-to-end testing tool for web applications due to a bunch of its powerful features, user-friendly interface, fast test execution time, easy installation and debugging, etc. To my mind, Cypress is a real game changer in end-to-end and component testing and it grows at a rapid pace. Among the many benefits of Cypress, I would like to emphasize the high quality of its , as well as the master classes that the Cypress development team conducts and publishes in the public domain, and also a very friendly and responsive . Honestly, and as you may have noted from my previous articles I’m a big fan of that wonderful tool! documentation community . According to official , Cypress has currently support for (including and Chromium-based ), (Safari’s browser engine), and . Excluding , any browser you want to run Cypress tests in needs to be installed on your local system or CI environment. Cypress has the capability to run tests across multiple browsers documentation Chrome-family browsers Electron Microsoft Edge WebKit Firefox Electron Obviously, it is often not necessary to run all available test suites in different browsers given the increase in test execution time and the associated cost of the required CI infrastructure. Therefore, for deploying cross-browser testing in CI pipelines depending on the needs of a particular project. Cypress provides different continuous integration strategies Benefits of Using Cypress to Optimize Cross Browser Testing in CI In order to balance costs and available CI resources, as well as the optimal level of confidence in testing, : Cypress provides the following options for effectively organizing cross-browser testing . For example, sometimes it makes sense to run all available tests within , but within test files using flag. In some cases, the priority areas for implementing cross-browser testing may be critical application features or workflows, as well as the most likely user scenarios. Selecting a specific test suite for a given browser Chrome Firefox _execute only the happy or critical path related test files, or a directory of specific _“smoke” --spec . For example, running tests within can be triggered by events in the repository, while within , tests will run on a set schedule based on release periodic frequency. Modern CI pipelines allow you to set the required time and periodic frequency for running workflows. Flexible setting of the periodic frequency of launching individual browsers Chrome Firefox , where the groups are based on the browsers being tested. For example, using allows you to run each browser at different levels of , differentiating the amount of allocated CI resources between browsers depending on the importance of each browser in the testing strategy. For example, tests within can be run conditionally in parallel on up to four machines, while within on two, minimizing CI costs. Parallel execution of test files for each group Cypress Cloud parallelization Chrome Firefox . Sometimes it makes sense to run or ignore one or more tests in certain browsers to shorten the duration of a test running. To do this, Cypress allows to specify directly in the configuration of a test or test suite a specific browser to run or exclude, for example: or . Possibility to configure the launching or exclusion of browsers for a specific test/test suite { browser: 'firefox' } { browser: '!chrome' } . In cases where the project shows consistently stable behavior in different browsers, it is advisable to set up cross-browser testing only before deploying changes to the environment. Determination of the software deployment environment production A thoughtful combination of these Cypress advantages will help you build an optimal cross-browser testing strategy based on the needs of a particular project. Why Use Docker in Cross Browser Testing? There are various approaches to implementing cross-browser testing with Cypress, one of which is to set up the automatic running of Cypress tests in chosen CI platform using . Docker Docker makes it much easier to set up and maintain a cross-browser test environment. By encapsulating the entire test stack, including different browsers in isolated Docker containers, it is possible to reproduce a stable and consistent environment for running tests in different browsers, regardless of the servers running the containers. Using allows us to effortlessly scale our testing infrastructure by distributing our workload across multiple containers. So, it is possible to define several containers for different browsers and run them simultaneously using one command, or rather one configuration file. Parallel execution of tests in different browsers obviously significantly speeds up the overall testing process. Docker Compose Using the with pre-installed browsers as a base layer eliminates the need to install browsers on servers running Docker containers. You can read more about the benefits of using Docker in testing in my previous about running Cypress tests in Docker containers. official Cypress Docker images article Moving from theory to practice… In this article, I use as my continuous integration platform. It should be noted that , based on which it is quite simple to organize various options for running tests in several browsers. For example, you can create a workflow for each browser separately and activate the created workflows depending on the specified trigger events in the repository. Obviously, this is very convenient. GitHub Actions Cypress provides many useful of configuration files for setting up GitHub Actions workflows examples This article provides a simplified example of creating multiple containers to run tests within different browsers simultaneously using Docker Compose. The idea is pretty simple — let’s say we need to run a specific Cypress test suite within four browsers — , and . Trigger events for automatically starting a workflow in the repository should be a event to the branch, an or , and a start, for example, every Friday at 2 am. Google Chrome, Firefox, Microsoft Edge Electron push main open reopened pull request scheduled Also, during the execution of the workflow, it is necessary to obtain with the results of running tests in each of the browsers — videos and screenshots in case any tests fail. artifacts One possible solution to this task is to create a custom Docker image based on one of the , build and simultaneously run four containers from the created image, in each of which our Cypress tests will be executed in a specific browser. official Cypress Docker images Briefly about the test project To demonstrate how to run Cypress tests across multiple browsers, I use the very simple project to test my blog on Medium. The project already has some dependencies installed — and , and also has spec.cy.ts file with a set of three trivial tests for the blog’s homepage: Cypress-Docker “Testing with Cypress” Cypress 12.13.0 Typescript 5.0.4 By running it locally within browser, we make sure that all tests pass successfully: Chrome Obviously, these tests are only demonstration examples and do not indicate the correct display and functionality of the website in different browsers. To run tests on the GitHub Actions platform, this project is hosted on GitHub. Building a Docker image As a base layer for building the image, the official Docker image was taken, which includes all operating system dependencies and some browsers. Cypress/Browsers The most current version of the image at the moment — includes pre-installed , as well as three browsers — and . Given that the browser is pre-installed in Cypress, we will have all the necessary browsers at our disposal to carry out cross-browser test execution in accordance with the initial task. node-18.16.0-chrome-113.0.5672.92–1-ff-113.0-edge-113.0.1774.35–1 Node.js 18.16.0 Google Chrome, Firefox Microsoft Edge Electron The final for building the required image will look like this: Dockerfile Firstly, instruction defines a base image, all of whose dependencies and configurations will be included in the generated image. FROM The next step creates the working directory, in which all subsequent commands will be executed. WORKDIR /e2e Next, in instruction the and files, as well as the folder, including the file, will be copied from the repository to the working directory inside the image. COPY package.json cypress.config.ts cypress spec Then instruction specifies two commands to run, in particular, — to install the necessary dependencies in the working directory of the image and — to display information about Cypress, current browsers detected by Cypress, and so on. RUN npm i npx cypress info The last step in instruction the command in form is defined to run Cypress in headless mode in containers generated from this image. ENTRYPOINT exec Setting up running containers with Docker Compose As mentioned earlier, after building the Docker image, containers will be created based on it. In each container, Cypress tests will be run in a specific browser. To simultaneously run containers with a single command, it is advisable to use a tool such as . To describe the process of loading and configuring containers, the YAML configuration file should include the following: four Docker Compose As we can see, in the configuration file we define (containers) — , , and , each of which uses the Docker image created based on the located in the same directory ( keys). four services e2e-chrome e2e-firefox e2e-edge e2e-electron Dockerfile build Next, the values ​​of the keys set the commands to launch Cypress in browsers with the given names. It is worth noting here that in the service, the command was supplemented by a configuration change due to an open with recording video when using the browser in Cypress. command e2e-firefox issue Firefox The next step is to mount volumes ( keys) for each service and set the appropriate mappings to gain access to artifacts outside of containers. In essence, this means that the videos and screenshots generated during the launch of Cypress and written into containers will actually be stored in the GitHub Actions virtual environment at the specified paths relative to the workspace of the launched workflow. This will allow, in case any tests fail, to extract artifacts from the directory during the execution of the GitHub Actions workflow, which is required according to the initial task. volumes ./artifacts Setting up the Workflow in GitHub Actions To run the workflow, we’re going to use following workflow file which is located in directory of the project: e2e.yml .github/workflows Let’s analyze it in more detail. By using key we define trigger events that automatically activate the workflow according to the initial task conditions: on — when pushing changes to the branch push main — when or pull_request opening reopening a pull request — run a workflow every Friday at 2am schedule scheduled Next, we define one job in the workflow, . The job will run in a virtual machine with the latest version of the operating system within the configured of to ensure that an accidental hang does not use up extra CI minutes. cypress-run Ubuntu Linux timeout 5 minutes The key combines all the necessary steps to complete the task. First, custom application is launched, which extracts the repository into the virtual machine, sequentially performing the necessary actions, including checking the version, creating the necessary folders, authorizing, etc. steps actions/checkout@v3 git Next, we see the main step of the job — , which executes command to build and run four Docker containers with the specified names ( , , and ) defined in Docker Compose configurations. In this case flag means running containers in the background to allow the workflow to continue executing subsequent steps. Run docker-compose docker-compose up e2e-chrome e2e-firefox e2e-edge e2e-electron -d The purpose of the next step is to provide visibility into the logs of running Docker containers in order to monitor them and control the process of starting services before moving on to subsequent workflow steps. The command displays logs of all running containers in real-time. At this step, it is possible to track the execution of Cypress tests step by step in each of the browsers. docker-compose logs -f four The next four steps are identical in nature and ensure that the workflow artifacts from each container are downloaded for access after the workflow completes. The custom application takes the paths provided as input and uploads folders with generated by Cypress videos and screenshots if the test fails. This will make the artifacts from each container available in the workflow summary. In addition, the behavior of the action in case no artifact files are found is also configured here, as well as the storage period for artifacts ( ). actions/upload-artifact@v3 5 days Starting the Workflow To activate the workflow, let’s create a . For this let’s make a small change to one of the existing tests, for example, add “.” at the end of the expected header text (highlighted) for the test to fail: trigger event Next, let’s commit and push the change to the branch: main Voila, the workflow is started! Let’s go to the project repository on GitHub and open the summary of the last workflow in the tab: Actions In the log of the completed job, we make sure that all steps were completed successfully: cypress-run In particular, at the step, a Docker image was built based on the previously described , from which four containers were generated. During the build of the image, command was executed and information about detected browsers was displayed in the log, as well as other characteristics of the test environment — the operating system, versions of Node.js, Cypress, etc.: Run docker-compose Dockerfile npx cypress info At the next step, processes were simultaneously launched in the created containers: to be more precise, parallel execution of Cypress tests was launched in four browsers: We can see logs from each container about the progress of the test execution: As a result, the first test failed as expected, while the next two passed successfully in each of the browsers: four Validate Workflow Artifacts As you may have noticed earlier, the workflow summary contains information about the downloaded artifacts: After downloading the artifacts from the workflow summary in ZIP format, we verify that we have videos and screenshots of the failing test in each browser (except the video in ): Firefox Let’s check that in case of successful completion of all tests, the workflow artifacts will not be loaded. To do this, let’s fix the first test, make a commit, and push the change to the branch again: main After the workflow is completed, we make sure that artifacts were not loaded from any container due to their absence: Final thoughts In conclusion, it should be noted that the optimization of cross-browser testing with Cypress based on the approach described in this article has a bunch of obvious advantages. In particular, the are guaranteed. Containerization ensures that the when running Cypress tests on different browsers. simultaneous launch and parallel execution of tests in several browsers test environment is consistent and reproducible Moreover, it is based on a single configuration file. The depending on the desired level of parallelism for each run or the number of browsers being tested, etc. easier to set up and maintain a test environment test infrastructure scales easily In general, all this allows you to . increase the efficiency of using available CI resources, reduce the total time for executing tests, expand test coverage in several browsers, providing the optimal level of reliability, taking into account the specifics of a particular project That’s about it. If you found this useful, share it with a friend or community. Maybe there’s someone who will benefit from it as well. To continue your journey with me and get more information about testing with the awesome tool, you might be interested in subscribing to my blog and get notified when there’s a new useful article. Cypress “Testing with Cypress” The source code of all examples presented in this article can be found in the of the blog on GitHub. repository Thank you for your attention! Happy testing! Also published . here

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Conducting Cross Browser Testing With Cypress in CI/CD using Docker

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Cypress Testing in Docker: Different Docker Images for Seamless Testing

104 Stories To Learn About Continuous Integration

139 Stories To Learn About Cicd

15 of the Best Continuous Delivery Tools

5 Best Microservices CI/CD Tools You Need to Check Out

53 Stories To Learn About Continuous Deployment

Cypress Testing in Docker: Different Docker Images for Seamless Testing

104 Stories To Learn About Continuous Integration

139 Stories To Learn About Cicd

15 of the Best Continuous Delivery Tools

5 Best Microservices CI/CD Tools You Need to Check Out

53 Stories To Learn About Continuous Deployment

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps