Software Development is Not About Unit Tests

Photo by ShareGrid on Unsplash

Testing is essential but it's secondary. Sorry.

Things Fall Apart

I last worked onsite in 2010 for a company in Seattle and had my first new exposure to the software industry last September.

A lot has changed, and so far as I can see none of it for the better; there is a weird emphasis on programming as a social activity, like teamwork. And while yes most software projects are too large for one person, working more than occasionally in the physical presence of others destroys the concentration and focus that are prerequisite to high-quality software development.

But the weirdest change is that unit tests are more important that writing code.

Full of Passionate Intensity

Reading about software development here, in other online forums, in industry journals, anywhere, the emphasis has shifted away from actual deliverable software; the real star of the show is writing tests. I wrote on Medium about formatting and someone responded that she’d rather discuss what to test. All the passion and no small amount of thousand-mile-stare fanaticism are about unit tests.

It’s undeniable that in the past we didn’t take testing very seriously. For Microsoft testing was another box to be checked; some QA departments were very diligent but they were never given enough time to be thorough and recently Satya’s Vision© includes laying off most of the company’s SDETs. The standing joke was that customers do the testing and their bug reports result in some fixes in the odd-numbered release versions.

I think we are now seeing an overreaction to this lack of QA rigor, spurred by the faintly ridiculous approach called Test-Driven Development. Anyone who has ever actually written software and allowed himself to learn anything can see the core flaw in TDD; we always learn things during development that we didn’t think of in design so writing tests before implementation guarantees that either the tests will not be complete or they will be continually revisited.

A reference is better than a repetition, two references for good measure

TDD is Fundamentally Wrong Part I

TDD is Fundamentally Wrong Part II

Surely Some Revelation is at Hand

Development is Secondary

If I had a nickel for every time I read someone saying

There is nothing more important in software than the unit tests

I could buy a pretty nice CD box set.

This is simply untrue. It’s important that we test our own work, but it’s a lot more important that someone else tests it, someone unprejudiced by knowledge of the code and by being too close to the problem.

This frolicful notion extends to the point that if one is pressed for time to meet a deadline, it’s better to cut some corners in the coding than in the testing. That’s crazy.

We Don’ Need No Stinking Documents

This is an authentically fanatic idea, that design documents are obsolete; the unit tests are the design; they are where the APIs are defined.

Nonsense. Software is defined by need. Whether it’s a piece of personal-use hobbyware like something to bulk-rename downloaded music files (why I learned WinForms) or a suite of business microservices, we write executable software in response to a need. Testing that software is secondary to that need; no, testing is not inessential, but if anyone thinks testing is more important than development then someone needs to get away from the computer for a while.

Seriously.

Devotion to TDD aside, very few developers like to write documentation, even fewer can write clearly and thoroughly. It’s a skill worth cultivating.

Backwards is Better

Inversion is a passion for many in software; doing things opposite an accepted standard is peculiarly viewed as some sort of insight. This nutty idea reaches its peak in code formatting.

Test-Driven Development advances the Bizarro World notion that writing tests to a completed design is inferior to writing tests to a pre-implementation guess.

If I had a nickel for every time I’ve heard someone say

When the tests all pass, you’re done

I could … oh, never mind.

If I Die, Avenge Me

Here’s some real Software Schmaltz:

Anything short of 100% test coverage is dereliction of duty, 100% coverage is a point of honor

This is some seriously fanatic faux-military junk. No, you don’t need 100% code coverage in your testing. Stop saying that and think about what you’re doing. For God’s sake, man. If your regression test exposes a bug you’re going to read your tracelogging if you’re smart and otherwise you’re going to run it through a debugger and see that the failure is in some function that has a dependency it shouldn’t.

Write your tests for public entry points, period. You can’t write unit tests for a private function without making it public first. Which you should never do.

And this brings to mind another point about unit testing; unit testing a piece of code that works with runtime data requires the creation and maintenance of fake data sets, which themselves have to be tested and regularly revisited to verify their validity.

In fact, the closer one looks at unit testing, the better good old-fashioned BlackBox testing looks.

One Pair of Eyes is Enough

If I had a nickel for every time I tested my own code to satisfaction and then saw a coworker break it in under a minute, I could buy a private jet.

The nuttiest idea I’ve run across (outside of writing the tests before the implementation) is the one that says developers are solely responsible for testing their own work, that we don’t need BlackBox testers anymore. We don’t need a second pair of eyes.

This is self-evidently absurd to anyone with any experience; every one of us has blind spots, cases we overlook, and we will overlook the same cases writing tests that we overlooked writing code. This is elementary, obvious, not at all subtle or revelatory.

I hired a guy to test the company’s server code. He had no experience in developing or testing software, but he knew Google Sheets, and I added a few columns for him.

He found bugs.

Sure, some of them were misunderstandings of how the software worked but even so he found a lot of real bugs in software that had been extensively tested by its developers as well as by the company CEO. With no testing experience.

An experienced SDET is worth his weight in gold in the egg he will keep off your face. I don’t care if it’s manual testing, unit testing, or just ad hoc try-it-and-see-what-happens testing. Someone other than the developer needs to design and run tests.

Honestly, I don’t like unit testing, it’s a lot of work. and I especially dislike the idea that the demands of unit testing justify sloppy work in development, even if you sugar-coat it with verbal burble like “technical debt.”

On the other hand, if you find yourself working with a good BlackBox tester don’t be surprised if you spend as much time in his office as your own. These are the best coworker relationships I have ever had.

The Darkness Drops Again

I don’t think I need to point out the fanaticism of these attitudes. Yes, we didn’t do enough testing a generation ago. That doesn’t mean that testing is now more important or that plainly flawed ideas like TDD are justified. The problem with fanaticism, and there is no doubt that this new preoccupation with testing is loaded with it, is that fanatics don't let themselves learn. They will ignore any evidence that something isn't working, in fact that evidence will harden their determination to believe.

Run some tests, run some error cases, and hand it off to someone whose job it is to find bugs. Just don’t let him find any in the core functionality cases, because then you’re just being sloppy.