Translating Unix Philosophy into modern environments

Simply put, the Unix style is a set of tools to compose small units to form complex operations. In this sense it is much like functional programming, but living in application rather than theory. The two operations most notable from this era are that of configuration and concatenation of operators.

Take for example, a program to find source files that have tabs instead of spaces

find . -name test.py -print -exec cat {} \; | egrep './|\t'

This construction is mostly superfluous, but the point is that the developer has many options to both configure existing operators, and rope in new operators as necessary. In one line here, there is a directory search, an argument substitution, function concatenation, and search by regular expression. This sort of code would normally take a ~10 line program. Which may not sound so bad, but if you consider the thousands of such short programs that a Unix user might write in one day, the shell becomes much more pleasant to work with.

Functional style on the other hand, has much more in the way of composability but entirely lacks the configuration part (to the point of all named operators allowing only one narrow meaning). Modern environments typically have access to the Unix toolset, but choose not to use it. Why is this?

A first reason not to use Unix in applications is security. The control of a terminal is typically so powerful that any app invoking exec calls at any level is considered bad taste. Unix is for sys-admins, certainly not web apps.

Second, Unix has a high learning curve. All of the commands and options are obscure and often counter-intuitive. If there is one commonality among operators, it is simply the response to read the man page.

Third and finally, Unix is not very composable. Unix permits only text to text composition. Due to this intentional constraint, complex data-structures must be reduced to JSON etc. or otherwise become mired in application logic. There is a reason that applications turn into monoliths and it’s typically not for “performance reasons”.

So what is the way forward?

Well first, let’s hold ourselves to the previous standard. Let’s build a better sh. What would that look like, given today’s perspective?

If Unix style is not the way forward, then what is? If we look to Academic theory, then we might want to try Functional. How about Haskell or ML?

Since I am slightly more familiar with the ML branch, let’s try that. OCaml includes a REPL so if you want to follow along, start by installing and running ocaml in the shell.

Datatypes? check.

# type 'a btree = BTree of 'a btree * 'a * 'a btree | Empty;;

type 'a btree = BTree of 'a btree * 'a * 'a btree | Empty

# BTree(Empty,1,Empty);;

- : int btree = BTree (Empty, 1, Empty)

Pipe operator? check.

# let ( |> ) x f = f x;;

val ( |> ) : 'a -> ('a -> 'b) -> 'b = <fun>

# let f x = x+1;;

val f : int -> int = <fun>

# let g x = x+1;;

val g : int -> int = <fun>

# f 1 |> g;;

- : int = 3

Performance and safety? Comparatively check and check.

So we’re down to two things that we lack, configurability, and builtins.

For builtins, Haskell has a library called Turtle, which does a good job of porting as much of standard shell operators as possible. I don’t know of a similar project in OCaml. The result is slightly awkward, and probably not an improvement over the old shells.

For configurability, none of the includes do very good with parameterizing the operators. Haskell would not permit the kind of terse syntax for this that I would expect of a shell. Though OCaml supports optional parameters on functions, the outlook is not much better. If we were going to take full advantage of the datatype advantage, then we would need something like ad-hoc polymorphism. That feature, though generally thought to be helpful, is not widely supporting in the Functional Community, due to incompatibilities with how type inference is done.

Taken together, I have an optimistic outlook that the Unix philosophy will survive and transfer it’s style into modern language and shell use. However, for a while don’t expect to throw out all of the code that has survived these last 40 years. Good ideas are stubborn to discard.