How to Squeeze Test Driven Development on Legacy Systems

We all love T.D.D. We know its benefits, we have read a thousand tutorials on how to build a system using this technique. But this not feasible for currently legacy systems. What is TDD? (TDD) is a software process that relies on the repetition of a development cycle. Test-driven development DEVELOPMENT very short We turn requirements into very specific test cases. We improve software so all tests pass. This is opposed to incorporating functionality that has not been proven to comply with requirements. Created in 2003 by Kent Beck (also the testing system author). xUnit testing Framework The Cycle 1) Add a test ( ) it must fail Solely based on behavior. Forgetting everything about accidental implementation. 2) Run all tests. The new test must fail. All the rest should pass. 3) Write the simplest possible solution to make the test pass. The programmer must not write code that is beyond the functionality the test checks. ( and design principles) K.I.S.S. Y.A.G.N.I If the all test passes restart the process or ... 4) (optionally) make a refactor (when code stinks). NEVER DO BOTH 1 and 4 Together. How to Find the Stinky Parts of Your Code (Part I) Design Benefits Testability and better class interfaces Simpler designs (KISS, YAGNI, Gold Plating avoidance, , ) Fake it till you make it Fail Fast Isolation on failures (less debugger or logging uses). Design by contracts. Modularization Bottom up building. Normal Use cases and Exceptions (Alternate cases) separation. Full branches coverage (we cannot add code without a test covering it). Instant feedback / psychological rewards. Small steps incremental approach. Based on Wittgenstein learning ideas by incremental examples and Cognitive Behavioral Therapy. Defer implementation issues and Premature optimization. Requirements Test must be in . full environmental control No Globals, No , , No Database, No Caches, No External API Calls and no side effects at all. Singletons No Settings TDD can detect problems. coupling Solving them leads to cleaner code focused on business logic alone and encapsulating . implementation decisions We must deal with coupling problems using : , stubs, fake objects, spy, proxies, dummy objects, etc. test doubles mocks Coupling: The One and Only Software Designing Problem Working on existing systems According to the popular myth, we can't use TDD on existing systems. This is not true. Let's show an example. How to Decouple a Legacy System The real world example We have a ticketing system needing to showcase several artists performing live-streaming during COVID-19 pandemic. Users can search for artists based on a type-ahead selector on a React application. System performs database queries on a heavy concurrent back-end system. We need to remove SQL queries matching part of artists names. redundant is very expensive on relational systems, and we are not allowed to change back-end architecture. Like SQL Operator The Problem We need to simplify redundant searches. That's all. * ARTISTS ((artist.fullname ) (artist.fullname ) (artist.fullname ) (artist.fullname ) (artist.fullname ))) SELECT FROM WHERE LIKE '%Arcade Fire%' OR LIKE '%Radiohead.%' OR LIKE '%Radiohead%' OR LIKE '%Sigur Ros%' OR LIKE '%Sigur%' System will execute just: * ARTISTS ((artist.fullname ) (artist.fullname ) (artist.fullname ))) SELECT FROM WHERE LIKE '%Arcade Fire%' OR LIKE '%Radiohead%' OR LIKE '%Sigur%' Since this part is redundant and expensive to the database. OR (artist.fullname LIKE '%Radiohead.%') OR (artist.fullname LIKE '%Sigur Ros%') Let's get to work Always start with the simplest problem 1) Add a test (empty case) Notice: Class is not created yet. LikePatternSimplifier No function is defined. simplify() We number tests according to definition order. First test is the easiest one and also the of Zombies methodology. Zero Case Zombie Testing: One Behavior at a Time Test fails (as expected). Let’s create the class and the function. 3) Write the simplest possible solution to make the test pass. Notice: First solution is always hard-coded. Continue with another trivial case 1) Add a test (simple expression). 3) And the simplest solution for both cases. Works like a charm in both cases. We are taking the problem and following principle. baby steps, slicing divide and conquer Continue with another (not so simple) case: 1) Add a test (two independent expressions). Code works correctly without changes. Is this a good test? We will discuss it on a more advanced article. Let's move on. Continue with a desired and juicy business case. 1) Add a test (one expression containing the other). Let’s make it work. 3) Add the simplest solution for all the already written cases. This is an ugly algorithmic solution, but we will improve it with a refactoring once we become more confident. We cannot anymore. We need to . fake it make it Ugly, not performant, undeclarative and complex. We don't care. We need to gain confidence and learn on the domain. Luckily, we will soon have time for better solutions. Continue with another case. 1) Add a test (left expression containing the right one) … and we are Green, so we are covering the business rule stating that terms order is not relevant. (Commutative Property). We make it explicit so no smart refactor can ever break it! Move on with another (not so simple) case. 1) Add a test (Capitalization is not relevant to MYSQL engine but our users might not be aware of that) 2) We run the tests and is broken. Let’s fix it! the new one 3) The simplest solution for all the already written cases (with the new case). … and tests are all green again with the ugly improved solution. Code smells and we have several test cases. We need a better solution. How to Find the Stinky Parts of Your Code (Part II) 4) Let’s refactor the solution with a more efficient and readable one Let’s test first production scenario requested by customers. 1) Add two unrelated redundant prefixes And it works ! We inject it on our legacy code: Before Let's inject it. What really happened Up to here, we worked in isolation scenario. Software development is a group activity. The quality assurance engineers found additional possible benefits. Pattern could be in the middle of the string. Customer agreed to add this functionality. Lets consider those cases New cases are broken since they were not represented by a previous one. We keep fixing them. 4) Let’s change the solution to cover all previous cases and the new ones. The end is the beginning TDD works in all stages. Using CI/CD codefix went into production. Happy ending. Once we submitted the SQL something bad happened: intelligent simplifier This was actual SQL after terms of bad handling: * artists ((())) SELECT FROM WHERE This SQL generation mistaken as an empty condition. So we will fix it TDD Way. We isolate the defect and add it as a broken TDD Case Of Course, it fails since previous implementation brought an empty solution (and a customer complaint). We can fix it by doing a duplicate's remover case-insensitive pre-processor at the beginning of simplify function: To see if we must test private methods please visit shoulditestprivatemethods.com Tests are green again. Not dealing with case-sensitive duplicate's algorithm worked again. Lets consider a different order. Against our intuition we see it fails. This is because the unit is bringing a ‘Yes’ instead of a ‘yes’. The solution depends on the product owner. We can: Normalize all outputs. Change the test based on the property that our is case-insensitive on text fields. SQL Engine We choose 2) Tests are green again We add more tests considering mixed cases Missing Opportunities Tests worked with the new Solution and given expected SQL. Case went to peer review. One of the reviewers asked about comparison finding an improvement opportunity. not like We asked our customer-on-site for agreement. If user chooses not to see ’ => it is choosing not to see and . ‘head ‘Radiohead‘ Talking heads In SQL: NOT LIKE ‘%head%’ implies NOT LIKE ‘%Radiohead%’ which is redundant in an AND condition. Our was already aware of that, so we injected in a second place being confident tests were already covering that scenario. simplifier Conclusions Quality Assurance engineering should add a broken integration test before correcting the implementation that should comply with it. Implementing on a big system requires special techniques to gradually remove coupling. TDD influenced all the written code. We have all the new code covered. (17 unit tests and 3 SQL Generation tests). We gained confidence on every new case add ensuring they didn’t break previous ones. We should using method objects/FunctionAsObject or reflection. only test private methods We used TDD on Development, Code Review, QA fixing and Production Defects Life-cycle. We developed a parallel customer system (the tests). They will always be our first omnipresent user. Solutions were as simple as possible. It’s to make TDD on existing projects with lots of code. very possible TDD does not replace or overlap QA Process and tasks. Multiple roles were involved and added value: Developers, QA engineers, Customers and Code Reviewers. We faked it until we made it. TDD a good design. does not guarantee We should never not covered code. change or optimize IF you like this article please I can write more on TDD on Legacy systems. let me know, so Legacy Code is all the code without tests. Michael Feathers. Read More: 40+ Thought-Provoking Software Engineering Quotes Part of the objective of this series of articles is to generate spaces for debate and discussion on software design. We look forward to comments and suggestions on this article.