The Job Market of Software Plumbers is Overheated

This is a response to Karl L. Hughes’ The Bulk of Software Engineering in 2018 is Just Plumbing.

So we’re here, living in a world where most “software engineering” is essentially plumbing. What should we do about it? What does it mean for our careers? Will the money keep flowing forever?

It just so happens that I’ve been thinking about these questions for quite a while and I think I have some answers now.

1. Do the other part of software engineering.
2. You would have to become better or obsolete.
3. It depends on the 2.

Let me describe my week to show what I mean by “the other part”. The last week I was finishing a texture generating feature for 3D-printing related desktop application.

Monday was a refactoring day. I did some recomposition, removed an unused structure, got rid of some unnecessary type conversions. Suddenly, performance tests failed. Fascinating! Apparently, when I changes bitmap dimensions from unsigned int back to size_t , the rescaling algorithm lost some 30% in performance. Suddenly, Monday became disassembly day.

It turns out, the conversion from 64-bit size_t to 32-bit int helps the compiler exploit the register space optimally. When everything fits the registers, the rescale runs super-fast, but when not, it has to upload a constant bitmap dimension every time in the tightest spot, which ruins the perfection. If not the performance coverage and disassembly, I would have never figured this out.

Tuesday morning I had a meeting with our application engineer. We discussed an interpolating function for the generator. I proposed a cubic polynomial and, so see and show how would it behave in our case, made one of the wordsandbuttons’ pages into a small HTML+Javascript demo, and sent it to him immediately.

He replied just an hour later. The cubic works for the most scenarios we want, but it would be good to have options. This made my Wednesday the SymPy day.

I could have taken any NURBS library to make a spline instead of a cubic, but since it was meant to work in the tight spot, and NURBS is an overkill anyway, I decided to hand-craft a special non-uniform non-rational 2–3–2 polynomial spline. That’s what I needed SymPy for. When you have your constraints clarified, all you have to do to make a spline — is to solve a few Vandermonde systems symbolically — and this will be basically your code. It takes some effort, but it doesn’t bring a third-party dependency and, since you’re not bound to any predetermined data-structures, the compiler works wonders with the optimizer.

Thursday was the finalization day. Full testing, final code review, gated check-in. Gated check-in failed. Turns out, I forgot a standard mismatch between VS compiler and an XCode’s Clang. One man’s habit is another man’s syntax error.

Friday afternoon I taught in the Academy. I told about this size_t to int thing. Shown how to read a disassembly. Explained floating point error and why do we want stability in our algorithms.

A student shown me an interesting thing in return. When an initializer list uses the static class member, the latter may not be initialized yet. I would expect it to be stored in the .data and thus fully initialized by the time of object construction in the runtime. Apparently, I was wrong.

Well, I would never do teaching if not for the opportunity to learn.

90% of our users would have been happy with the initial performance. 90% would have settled for linear interpolation. 90% don’t care about macOS port. None of the users care about our students. So was it all worth the effort?

Yes.

It is a dangerous misconception that business is about achieving goals at the lowest possible cost. Business is about competition, and “good enough” doesn’t win market share. Everyone can do 90%, you have to do better than that. You have to be better than your rivals and you have to be better than your former self as well.

Investors are willing to invest. If you are willing to walk an extra mile, the stakeholders will walk it with you. As long as you are explicit about the cost and benefits, of course.

Business doesn’t want our job to became boring and miserable. Quite the contrary. Business is as challenging as ever. Robustness, performance, and scalability are all in high demand right now. Probably higher than a decade ago when you could have covered your butt with the Moor’s law.

And I totally agree with Karl in his last advice: don’t get comfortable. Job market of software plumbers is way overheated indeed. And it takes a whole lot of extra miles to come to the world of software engineering.