My CTO: “We’re really impressed with your testing solution, you’ve gone above and beyond!” (Immediate feeling of guilt washes over me) This happened a couple months ago after my heard about a testing solution I put together for our web applications. My solution was to auto-generate the QA Selenium code instead of manually coding it by recording/configuring the tests. (Shameless plug: it’s available for free over at ) . This tool has dramatically increased coverage - but why did I still feel guilty about accepting this compliment? CTO snaptest.io It’s because I unreasonably assume testing should be exhaustive. refers to running a test on every possible combination of ns/data. Exhaustive testing: actio The comes when I say things like: “I’ve tested the new feature”, or “the tests are passing”. In the back of my mind, I know there are hundreds of thousands of combinations I didn’t test for. guilt We must give ourselves a break and remember that is usually not possible. One must reconcile the cost/reward balance sheet of testing… Especially when you start thinking about exhaustive testing negative tests. refers to checking how gracefully a system handles errors or unexpected situations. This is when exhaustive testing gets exhausting… Don’t expect to get them all. Negative tests/Destructive tests/error path testing — Here are my recommendations for handling negative test cases: Pick off the easy negative tests like form validation. Defer negative tests that will take a lot of work to create and maintain. Such as slow network connections on mobile applications. It may be more economical just to manually test those occasionally. Spend time making negative tests on the most commonly experienced errors. Ask yourselves, which errors are users most likely to see? Cover those ones. Don’t worry if you don’t cover every state of the app. Just remember, any test is helpful!
