How to set up E2E browser testing for your GitHub project

Nowadays, a vast number of projects, small and large, is developed using tools and hosted on . Authors of these projects often face the need to choose a tool for end-to-end testing. When we at DevExpress created , we wanted it to perfectly fit in the modern development workflow. Node.js GitHub TestCafe Node.js The same language is used both in tests and projects you are developing — modern with support for ES6 and ES7 features. JavaScript Easy installation and launch. No need to install and maintain additional products and browser plugins. Tests can be run right after . npm install Easy integration with CI systems. TestCafe functionality can be extended with plugins. Tests can use any third-party Node.js module. In this article, I’ll show how easy it is to set up automated testing for your Node.js web project on GitHub. Using a real project ( ) as an example, I’ll describe how to set up testing in local browsers on a developer’s machine and how to configure Continuous Integration testing on step by step. In the CI testing part, I’ll show how to run tests in the browsers provided by Travis CI (Chrome and Firefox), headless browsers (the browser automation library) and in cloud services that provide a wide range of operating systems and browsers for testing ( ). Bootstrap Travis CI NightmareJS SauceLabs I’ll show how end-to-end testing can be used at each stage of the project’s workflow: On a local machine during the development. You can run tests in any browser without installing additional libraries and plugins. In a pull request. This allows you to ensure that the project works well before the changes are pushed to the master branch. When pushing commits to the master branch. To run tests, we will use testing framework. TestCafe We will write tests for , the most popular HTML, CSS and JavaScript Framework for developing web projects. To this end, I’ve forked the Bootstrap repository and created a new branch . When this article is written, the repository has unit tests based on QUnit that are run on sending each pull request and merging to the master branch. In addition, the repo contains used to test components visually, with manual verification by humans. We’ll write functional tests for one of these pages. Bootstrap testcafe pages Writing Tests We will use the pattern in our tests. This approach allows us to create more declarative tests. In this article, we won’t focus on test syntax. For information about test API, refer to TestCafe . Page Object documentation Let’s write a test for a page with Bootstrap’s Modal component. Modal Dialog internally contains a number of other components (Accordion, Button, etc). The will help us create a Page Object. We’ll describe the entire page and its components with classes in a separate file. TestCafe selector system page-model.js You can find the entire page object . here With the page object, tests do not have to work with HTML markup. Instead, they operate the object representation of the page. A complete version of this test fixture is available . here Testing in Local Browsers During Development To test a modal dialog page, you need to have an HTTP server that will serve up this page. You may already have a mechanism that serves tested pages in your project. I will use a simple, zero-configuration . http-server Install TestCafe and http-server. npm install -g testcafe http-server Immediately after this, you’ll be able to run tests in any browser installed on your machine with one line in the console. testcafe chrome,firefox js/tests/functional/*-test.js --app "http-server ./ -p 3000" The option allows you to specify a command that deploys the application you are going to test. --app TestCafe will start http-server, launch the specified browsers, run tests in them, stop the server and output the report. TestCafe can also launch a browser by the path to its executable (good for portable browsers), pass arguments to the browser launch command and run tests in remote browsers. CLI Setting Up Continuous Integration Testing We’ll use as a CI service. Alternatively, we could use other services ( , , etc) with different features. Travis AppVeyor CircleCI Travis CI provides Linux virtual machines with pre-installed Chrome and Firefox. At first, we should add the modules we have installed earlier to the project’s dev dependencies. You can do this by manually editing or by running the following command package.json npm install testcafe http-server --save-dev Setting Up Travis CI Let’s configure Travis CI to run tests for each pull request. First, you need to enable Travis for your project. to Travis CI with your GitHub account. Travis CI will synchronize your repositories from GitHub. You can see them on your . Sign in profile page Enable Travis CI for a repository you want to build by flicking the switch on. By default, Travic CI runs builds on pushes and pull requests. You can change this behavior in your repository’s settings. Let’s configure the file. In this file, we’ll specify the Node.js version and browser settings. In addition, since Travis CI uses Ubuntu Server virtual machines that do not have regular graphical environment, we to run browsers headlessly. .travis.yml set up and use Xvfb For details on Travis configuration, see its . documentation Run Tests in a Pre-installed Browser for a Pull Request To run tests in a browser pre-installed to Travis CI, add an command to with the line we used to run tests locally. npm test package.json Now when a pull request to the master branch is created, tests will run on Travis and you will be able to view a . The commit status in the pull request will change accordingly. test run report After the pull request is merged and changes are pulled to the master branch, Travis CI will run tests one more time. You can view their status in your Travis CI account or you can add a to your repository readme. badge Configure Tests to Run in a Headless Browser If you do not need to test browser-dependent behavior, you may find it useful to run tests in a headless browser. This allows you to avoid installing real browsers on a testing machine. You only need to install a single module with . For instance, a TestCafe community member created a that allows you to run tests in NighmareJS out of the box. Let’s install this plugin. npm install plugin npm install testcafe-browser-provider-nightmare --save-dev After that, you can pass as a browser name to the test run command. nightmare testcafe nightmare js/tests/functional/*-test.js TestCafe allows you to easily create your own plugin for any such library. You can find the instructions in our . documentation Testing in Any Browsers with Cross-Browser Testing Services If the testing options described above do not satisfy you needs, consider testing in real browsers with crossbrowser testing services ( , , etc). TestCafe has a that allows you to run tests on virtual machines on SauceLabs out of the box. A if you use it to test an opensource project. BrowserStack SauceLabs plugin SauceLabs account is free To run tests, first install a plugin. npm install testcafe-browser-provider-saucelabs --save-dev Then you need to specify the login and access key for you SauceLabs account in global variables ( , ). When running tests locally, you can do this through the console. To run tests on Travis, specify the credentials in your Travis account. SAUCE_USERNAME SAUCE_ACCESS_KEY TestCafe can show all browsers available via SauceLabs with the following command: testcafe --list-browsers saucelabs Let’s run our tests in desktop and mobile browsers: testcafe "saucelabs:Chrome@beta:Windows 10,saucelabs:iPad Simulator@8.1" js/tests/functional/*-test.js --app "http-server ./ -p 3000" TestCafe will set up a tunnel between your machine and the SauceLabs server, start virtual machines, run tests and output the result. If you are opening a pull request from a repository fork, there are certain limitations. It is not possible to do cross-browser testing in pull requests via Travis CI because this may potentially compromise your SauceLabs credentials. Fortunately, you can work around this by using . Savage Conclusion As you can see, configuring TestCafe does not require any difficult operations. It is made with just a single console command. TestCafe can output reports in different formats (json, nunit, spec, list, minimal) and allows you to create your own report format with a . This enables you to use TestCafe with every CI system. You can also run your tests with a or task. plugin gulp grunt Since TestCafe is written for Node.js and has both command line and programmatic API, you can configure the testing process flexibly to suit your particular tasks and use third-party Node.js libraries. If you are interested in TestCafe, you can learn more on our or . site repository Please use the comments section to tell if you have any questions when building CI with the existing solutions. Do you think TestCafe may help you? I would appreciate your opinions and feedback. And I’m ready to answer your questions.