Chapter Three in a very personal history of programming
If the previous two decades were the dark ages, the 1970’s were the Renaissance; a bridge between the classical era of computing and the modern era of programming. It was a time of great change for Western society in general and the world of programming was no exception. Programming was on the verge of becoming the activity we know today; it was becoming possible to write code without an intimate knowledge of the computer hardware. CRT based VDTs first introduced with IBM’s System 360 and originally costing as much again as the computer itself, became affordable and common. New programming languages changed the way people worked, and by the end of the decade, personal computers became available at reasonable prices.
The 1970’s are also when my narrative starts to become personal. Although I myself did not start to program until the eighties, the programmer culture and history of the seventies was still a very real and contemporary presence for me as I learned the craft. The people I write about, most of whom were heroes to me, were in the prime of their careers when I was just starting mine, and I have personally used most of the systems and languages that I will write about from this point on.
One system that was tremendously important was Unix™. It was created by a team of five programmers at AT&T Bell Laboratories and was portable from one hardware platform to another. Its distribution included the source code for the OS itself. For this reason, Unix™ was “ported” to many different hardware platforms. It was the beginning of operating systems that were independent of specific hardware platforms. It would be difficult to overstate the influence that Unix™ has had on computing in general, but more specifically on the craft¹ of programming.
From the perspective of personal computers (my own perspective), which initially ran crude little operating systems like MS-DOS and CP/M, Unix™ had the aura of “the big leagues”; rich in features, impressively flexible, at times arcane and inscrutable, at others powerful and simple. When I started working in Unix™ I felt I had arrived. To my bosses, I was elite. When GE Information Services was asked by GM’s Electronic Data Systems to setup a Unix™ server farm to control the computerized painting of Camaro automobiles at their Sainte-Thérèse plant, I was the one they sent. I knew Unix™! However, from the perspective of the real big leagues — operating systems for mainframes and minicomputers — nothing could be further from reality.
Unix™ was developed by a team working on a “real” operating system: Multics. In those days “real” operating systems (used by “real” programmers) ran on huge, powerful, extremely expensive computers. One of their jobs was to manage this expensive resource as carefully as possible. Therefore, significant portions of their code were dedicated to complicated user management, with accounts that had quotas or allocations of computer time, and code to remove time from the account as program “jobs” were run. They managed the separation between jobs, the “time-slicing” that optimized the use of the CPU, the communications protocols to terminals and disks and printers, and a long list of other things besides.
The name Unix was a joke, a play on words. Designed as a playground for developers, it intentionally lacked all the complicated controls of Multics, and so it was Un-Multics, it was Uni-user because it lacked what were considered standard multi-user features. But at the same time, the OS itself was a very serious endeavor. It was meant to be a productive platform for programmers to develop software and it had a single unifying philosophy: “the idea that the power of a system comes more from the relationships among programs than from the programs themselves”.² It had architectural coherence: terminals, printers, and even disks and network adaptors were all treated the same way through a single simple communications model.
And it was programmable.
When one purchased Unix™ from AT&T (for the princely sum of twenty-thousand dollars) it came with the source code. The implications of that were staggering for the time. Up until this time, operating systems were almost exclusively created by the computer hardware manufacturers for a specific computer or series of computers and were jealously guarded trade secrets. It was not only impossible — it was unthinkable — to modify the OS in any way. By the unorthodox inclusion of the source code, AT&T made it possible for any licensee to do two things that were completely novel: they could modify it, and they could compile it and run it on any computer that had a C compiler (which very quickly became quite a few), and that meant that a programmer could replace the original operating system that came with the computer!
Not that AT&T was completely free from the closely guarded trade secret mentality. Every time I have written Unix above I included a trademark symbol next to it. This is an in-joke for any reader who happened to have been a programmer in the 70’s and 80’s. AT&T was infamous for its fanatical pursuit of trademark infringement. Usenet lore had it that even a casual mention of Unix without the ™ would bring down a storm of corporate legal aggression so fearsome that it could take years off of one’s life. As such, many of us carefully, and very ironically, inserted (tm) after every mention of the name. I hope I gave a smile to at least one or two readers who remember those days.
So how did Unix help improve productivity in programming? Because of its philosophy of placing a premium on interactions between programs, Unix dictated a standard simple protocol by which programs would exchange data with other programs and with the computer and all its peripherals: text.
This sounds deceptively trivial. It is anything but. Anyone with even passing technical familiarity with Unix has heard the phrase: “in Unix, everything is a file”. What this means in practical terms is that a program does nothing differently at all to send the sentence “Hello World” to a printer, to the screen, to another program, or to a file on disk for storage because all of these things appear to the program as a file on disk. In fact, in traditional Unix usage, the program would not do any of these things itself. The user (who is also a programmer) would. The user would type the command HelloWorldProgram > lpt1 to direct the “standard output” (called stdout) of the Hello World program to Line Printer 1 or HelloWorldProgram | WordCount to pipe the output of Hello World to a program that would count how many words it received on its “standard input”. By standardizing the way programs exchanged data with anything outside of themselves Unix promoted a way of thinking that was to have a profound effect on the way programs were written. When you use your Facebook account to log into some other website, you are benefiting from the philosophy that Unix pioneered of different programs working together seamlessly. When your phone opens the map program to give you directions, or opens the email program to send an email, it is because Unix taught programmers that “the power of a system comes more from the relationships among programs than from the programs themselves”.
Because of this, Unix programs are often small, doing one thing only, such as tail, which reads in a file (remember: everything is a file) and sends the last 10 lines to its standard output, and it is normal for programs to be written with other programs in mind. For example: the Unix program man, which shows the user manual entries for various programs. The author of man did not bother to write code to break up long manual pages into readable chunks, because according to the philosophy of Unix, the output of man should be piped to another program such as either less or more, which are “paging” programs that break long files (remember: everything is a file) into pages the size of the screen, allowing the user to “turn” the pages by pressing the space bar.
Because of this philosophy, the programmer of man was able to leverage the previous work done by the programmer of less (or more). As you might imagine, this new approach to writing programs had a dramatic effect on productivity.
Smaller programs are easier to write, have fewer bugs, are easier to debug when needed, and are easier to maintain. If you clean your kitchen a little bit every day, it is a lot easier than trying one big cleanup every month. Programming is exactly like that, so writing smaller programs, that rely upon other previously written (and tested and debugged) programs, is more productive than writing big “kitchen sink” programs. Or as I call them: “world domination programs”.
This is a very important idea. It is a central concept to the book I am writing,of which this series is an excerpt. I deeply believe in the Unix philosophy that the best way to approach programming is by writing small program components within a powerful unifying framework, and then linking them together.
There was one more thing…
In 1972 the Unix team created a language to help them write their operating system. The language was called C. It is arguably the most influential programming language ever created. I could also argue that it set productivity back by a couple of decades.
<- Previous article in the series
Next article in the series ->
[1] Programming is both an art, as Knuth would have it, and a craft. The two are not mutually exclusive. When talking about creativity it is art. When talking about rigor and mastery it is a craft.
[2] Kernighan, Brian W, and Rob Pike. The UNIX Programming Environment. Englewood Cliffs, N.J., Prentice-Hall, 1984.
This article is an excerpt from my upcoming book The Chaos Factory which explains why most companies and government can’t write software that “just works”, and how it can be fixed.