Unit testing is utilized to examine small segments of code that can be secluded logically in the system. Frameworks normally managed to do unit testing are JUnit, NUnit, JMockit, PHPUnit, or tools like TestComplete.
The motive behind unit testing is to verify each software segment(unit)on the basis of expectation and the actual result we will get based on test cases.
Unit testing is the initial measure of testing in the complete software development life cycle. Unit Testing is basically performed by developers who are developing their code in a secluded environment and other testing measures come when developer’s code merges to master or deploys to a higher environment.
Unit Testing → Integration Testing → System Testing → Acceptance testing
Unit testing in the beginning itself will be cost and time effective in the end. It assists to settle bugs timely in the software development cycle. It helps coders to recognize the code functionality very easily. The test cases defined, should be self-contained and it's like testing one piece at a time. Your code attitude should be like that more code you put lacking testing, more locations developers have to verify for bugs.
Unit Testing can be done both Manually and Automation. But from the perspective of Software Engineering, automation testing is preferred. We will see how we can integrate unit testing with Jenkins now.
Doing unit testing in Jenkins - Freestyle Project
Proceed Manage Jenkins → Manage Plugins, then Install JUnit Plugin
Unit testing in Jenkins will be initiated after installing a unit testing plugin in Jenkins. This installation will enhance Jenkins' features for JUnit functionality. To notice the actual performance, let's observe it with a practical demo. We will examine it initially with freestyle projects.
- Click on “New Item” → Title the project and generate freestyle projects.
- During the configuration window, choose “Git” in “Source Code Management” section and supply all required details:
- Select “Invoke top-level Maven Targets” below the “Build Step” segment. (Note:- We are utilizing a maven project here for our example.
- Arrange and include the test step for the Goal text box. Let’s check each option
Maven Version -- If you have multiple versions of Maven installed, select one you preferred.
- Goals -- Maven goals, you want to incorporate with “mvn” command
- POM -- place of pom.xml file.
Now, once test records will be generated, we are required to execute some post measures on the basis of those records / reports. Pick “Publish JUnit test result report” below “Post-Build Actions”
- Test report XMLs -- Path where your XML files(test reports) are kept. Location defined in this text box is relative to the root of your workspace.
- Retain long standard output/error - If this choice is selected, output or failure from the test suite will be engaged in test results following the completion of build. This selection is essential uniquely when you desire to understand each and every log message in the console output of build but it will result in an increase in memory consumption of Jenkins. Not recommended
- Health report amplification factor - This is like an amplification factor, utilized when calculating test records presenting to build health score. Suppose, for “1”(default value), it will measure like 10% of tests lacking with an overall 90% health score.
- Allow empty results - If checkbox is selected, then it will change default actions of failing the build when there are misplaced or void test case files. It will not fail the build for any such structure, if and only if selected.
- Click Save & Build
- Verify the console output:
Running com.java.AppTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec «<< FAILURE!
testApp(com.java.AppTest) Time elapsed: 0.002 sec <<< FAILURE !
junit.framework.AssertionFailedError
at junit. framework. Assert.fail(Assert.java:47)
at junit. framework. Assert.assertTrue (Assert.java:20)
at junit.framework. Assert. assertTrue (Assert.java:27)
at com.java. AppTest.testApp (AppTest.java:36)
Results:
Failed tests: testApp(com.java. AppTest)
Tests run: 1, Failures: 1, Errors: 6, Skipped: 0- Test case, taken in this example, is a failed one, so that's why it broke the build. Improve your code so that your test case will be passed. Then Build again. Build will be successful now and observe that Jenkins dashboard will also be updated.
Click on “Latest Test Result”. Build job will be initiating a test report in this link. You will verify the number of test cases executed and failed, if any:
Similarly, you can observe a graph in your Jenkins job dashboard which will show you the course of test cases(failures/success) on the basis of jobs execution:
Previous build failed because of 1 test case failed, 2nd passed (0 failed test case). From the color contrast, you can easily guess which one is failed and which is passed. If you observe links inline with the chart:
- (just show failures) -- It will reveal only the movement corresponding to build where test cases failed.
- Enlarge -- As the title suggests, It will enlarge the graph to the whole window.
Unit Testing in Jenkins - Pipeline Job
- Now, we will test by adding unit testing steps in our pipeline. As most people don't want to proceed through thousands of lines of console output, the pipeline will issue a “junit” step to record and aggregate test results.
- JUnit will be defined in post step as build will be going through stages and following the completion it will produce the reports.
- Jenkinsfile “post” block will look like:
post {
always {
junit testResults: ***/target/ surefire-reports/*.xml', allowEmptyResults: false
//cleanws()
mail to: ‘ @gmail.com',
subject: "Build Job: ${currentBuild. fullDisplayName}",
body: "This is ${env.BUILD_URL}"
}
}
- Click New Item → Name the project and choose pipeline(or Multibranch as both will be using Jenkinsfile only) job. Press OK
- Configuration page will be opened. Define in the normal way by providing Github Branch Source to read Jenkinsfile from with proper credentials to link to repository. Click Save and then do the build.
Check Console Output:
[Pipeline] { (Declarative: Post Actions)
[Pipeline] junit
Recording test results
- Test results will be recorded and you will observe a new link appear on the left side with “Test Result”. Click on that result link to capture records. It will convey you in similar way, we observed in freestyle project:
Test result(Number of failed and passed tests)
Catalog of whole tests executed:
- Package
- Duration -- Time took to run
- Fail --- No. of tests crashed under that package
- Skip -- No. of tests Skipped
- Pass -- No. of tests passed
- Total -- Number of total tests accomplished
Similarly, you can dig into that package by clicking onto it to minute details from package to class to plain Test. In the right border of the “Test Result” segment, you will observe “Took 1.7 sec”. Press the link, a graph will be appeared displaying history of test records:
And you can debug each build test plot and compare.
Unit testing in Jenkins is executed by defining unit testing steps in Jenkinsfile with help of “junit” directive. Some organizations prefer to grab or store the artifacts for analyzing and investigating locally for any test failures. And for that we can utilize the “archive artifacts’ directive prior to “junit” step.
Process of Unit testing with Jenkins document test results which help in to surface details from Jenkins dashboard to the team that includes higher management also and can assist in planning deliveries.
More Options to do unit testing with Jenkins
There are multiple unit plugins available relying on the framework you are utilizing or want to utilize. Most of them are:
- AUnit
- CUnit
- CppUnit
- EmbUnit
- FPCUnit
- MbUnit
- NUnit
- PHPUnit
- UnitTest++
- xUnit.Net
If somehow, in an organization different projects, using different unit tests framework but Jenkins instance is same then you can install “xUnit” plugin froom your Manage Plugins section and utilise different framework options provided by Jenkins. I will give you a small demo of using this too:
Install xUnit Plugin, then you will see the “Publish xUnit Test results” option under “Post-build Actions”.
- After clicking “Add” in front of “Report Type”, you will get multiple options of Unit testing frameworks. Select as per your choice and proceed. I will select JUnit only.
- You can select multiple frameworks also in a single job (depending on the scenario). Thresholds section will give you the option to set the build status depending on the number specified in test cases. If I select “Failed Tests”
It will mark the build as unstable or failure depending on if new or total number of failed tests exceed the threshold provided by you.
- “Advanced” option will give you to configure it to minute details:
- Click Save and Build, you will get your xUnit recording your test results and work in a similar way like it was done while working with jUnit plugin. It will give you more power and control to specify your build status also.
Conclusion
Now, to summarise this blog, we discussed unit testing in Jenkins and unit testing with Jenkins with different types of jobs. This will help us in achieving one of the most effective parts of our SDLC. Next, we will discuss more about notifications in Jenkins. Waiting for your feedback while testing the testing plugin.
See you soon !!