Untold Stories About Python Unit Tests

Unit testing, if done the right way, will become your best buddy. The time you invest on writing tests will come handy in the most unexpected ways, once the product moves to a maintenance level. Unit tests will stay by your side as your pet doggy and bark at the silly mistakes you make during a feature addition, a refactoring or even a bug fix. Unit tests together with a static code analyser can prevent a majority of easy to make mistakes. Unit testing is your best buddy Pytest is undoubtedly the most popular and stable testing solution for python This article will share my experience on pytest, specially about unit and the stuff that I learned the hard way; or rather, about some stuff that most online articles or forums don’t talk about. testing Anatomy of a test Before diving into pytest, let me cover the basics of a unit test. As a practice, I mentally divide every unit test I write into 3 or 4 sections!! As a summary, first there will be few statements for . Then there will be some statements for followed by few other statements for . Occasionally as a fourth step, we may also need to the test. Let’s go through these sections in a bit more detail. setting up the test executing the actual methods doing assertions wrap up is important to . It is vital to mock all the of the function or method that is being tested. Otherwise, the test you write may pass at the first time, but not all the time. For instance, if the method performs an actual database operation, the test may pass when a specific value is present in the database, and will fail otherwise. This concept is similar to in functional programming, which return the same output to the given inputs no matter how many times you call the function. In fact, a unit test function can be thought of as a pure function itself. Thus the first part of a test method can be used to perform all the ground work for running a test properly.The library, , will help us in this aspect when writing unit tests for a python project. The first section “make the test a unit-test” side effects pure functions python mock which is now a part of the python standard library However in general, multiple test cases in a module require the same initialisation steps. To avoid rework and inconsistencies in such circumstances, we can use the methods to instruct the test runner to run the desired step before and after each test respectively. In some languages like Javascript, setup & teardown functions are known as and hooks. setup & teardown before after of the test can be used to call the actual method and perform any other required operations. Although this is not the place to write assertions, there are some instances where a test may yield the result here itself. For example, if you’ve ever worked with ReactJs, it is enough in some cases just to render a component. The test will not pass if the vital attributes of the component are not written correctly. This theory will apply for other non-react scenarios as well. However, it is always better to write some more assertions afterwards to validate the behaviour of the tested method or function. As per terminology, a test which does not do any assertions, but only checks for the execution without unexpected halts, is known as a . The second part smoke test , which is the most important part of a test, is for validating the behaviour of a function or a method. The assertions should not be too strict nor too flexible. Too much flexibility can hide the mistakes you might do during a code refactoring. Assertions being too strict can cause the tests to and fail even if the changes don’t cause issues. This will result in having to update the tests whenever you do even a minor code change, making the tests brittle, unstable and a nightmare to maintain. Deciding the ideal amount of assertions is an art which should be expertised with time. When it comes to python, assertions can be done with respect to the return values as well as through mocks if any. The third part over-fit On a separate note, it is quite easy to abuse this part to show increased test coverage. This is why the concept of mutation testing came into action. I will explain this concept in another article. The of a test is for wrapping up the test execution. For instance, if you mocked some method of a module at the start of a test, you might need to restore the original functionality. Otherwise, it can yield unexpected behaviour in preceding tests that use the same method. As mentioned above, this is a place where a teardown function can come in handy. However you might not notice this pattern when using mocks in python. That is because, when you use the decorator or context manager, the mocks will be automatically restored when the test ends. Thanks to this pattern, you rarely will need to use the setup and teardown methods in pytest. final and fourth part patch Pytest Basics Although Pytest is a great tool, I found their documentation quite descriptive yet hard to read. Therefore here, I will summarise the most important concepts from the original docs. Installation & Running You can install using and then simply run from the root directory to run all tests. pip install -U pytest pytest Test Directory & File Names Although there are many possibilities, the best place to define tests is, alongside the source file, in a directory called “test”. AKA “Tests as part of application code” The test files can be named either as or But please make sure you stick to one of these patterns. Here is an example from the original docs. _test.py test_ .py setup.pymypkg/__init__.pyapp.pyview.pytest/__init__.pytest_app.pytest_view.py... Integrating with Setuptools Most python projects are based on . If your python project is also using , you should integrate with the . setuptools setuptools pytest setup.py The pytest documentation clearly explains how to integrate pytest with setuptools using pytest-runner. Yet it does not explain why you should do that. When I had that question, I had no idea why I should do the said integration, when I could simply execute from the terminal. So I did the obvious; I skipped the integration. pytest Later on, at a time when I had just forgotten about this integration, I faced a problem. The source files show a coverage of 0% and all the test files are included in the report. I did not see that coming eh. The coverage report generated using pytest-cov is completely wrong!! After putting too much effort, I learnt that, unless you integrate pytest with the file, the tests will not run against the source as expected. Apart from that, there are few other reasons why you should integrate pytest with the . setup.py setup.py You can specify dependencies in the option in . This will ensure all required helper packages are installed before actually running pytest. tests_require setup.py You can alias pytest in the setup configuration file. For example, you can define your setup.cfg as follows and by doing so, pytest will run evertime before you create a source distribution using the command. sdist **[aliases]**test=pytest clean_all=clean --allrotate --match=.tar.gz --keep=0 build=clean_alltestinstallsdist **[tool:pytest]**addopts =--cov-config .coveragerc--cov-report html--cov=my_pkg--pylint--pylint-rcfile=.pylintrc--pylint-error-types=EF Also note, although not directly related to the setuptools script, how easy it is to define options to pytest using the file. setup.cfg Mocks As explained earlier mocking plays an essential role when writing unit tests in any language. The easiest way to mock methods in python is to use the decorator provided by the mock library. The main advantage of using is the automatic mock restore behaviour as explained above. On the other hand, a test written by making use of the decorator looks cleaner and easier to understand compared to the direct use of or . However, there can be rare cases where you need more flexibility in the mock objects. Just be cautious to reset the mock objects if you happen to use the core class directly. patch patch patch Mock MagicMock Every mock object created by the library have methods to do the required assertions directly. You can check whether a particular method was called and with what arguments just by calling these methods. You can directly specify a return value to a mock method. But you can also specify a side_effect method, which will be called instead of the original method. This concept is useful when you want to return some value based on the arguments passed when calling the method. Special Return Values When writing tests to methods that use external libraries, the return values may be special objects. For example, the return value of an actual call to the library is a special Response object. request In such scenarios, I made my way out by defining fake classes. As in above example, I configured the return value of calls to the request library by defining a fake Response class. Given below is a very basic example class like that. Thus now I can define the return value of a class as follows, mock_req.return_value = Response(status_code=200, data="abc123") Although there are many useful features provided by the pytest and mock libraries, the basic features will be enough for most cases. Nevertheless, the time you invest to read the documentation of a library will never go in vain. You will surely discover features which can do the same stuff that you already do in a better way with less code. Thus I strongly recommend you to at least glance through the pytest and mock documentation once you have grasped a some understanding about the basics. If you found this story interesting, encourage me to write more by giving some claps to this story. 👏👏👏 And I would love to hear your feedback through comments.