I have poor command-line hygiene. When I fever-dream a useful pipeline into existence, I very seldom commit it to a shell configuration somewhere. Rather, I let dredge it up sometime later when I need it. My functions are not documented, have hideously short names and variables because I tend to code golf, and are specific to the use-case-at-the-time and not general. fzf I’ve since decided that this is the optimal approach. Take That Statement Back, You Deviant No! I actually believe this. I have reasons. The notion of a command or shell history in our profession is a profoundly under-appreciated mechanism. Consider the fact that – if you retain history and express of your operational tasks in the form of terminal commands – you may have the entire history of your work (apart from coding artifacts) within a few seconds’ reach. How do you create x509 certs with ? What’s the easiest way to check for among a fleet of hosts? If you can recall a few substrings from the commands that may answer the question, a history search can bring this lost memory to your fingertips to be re-used immediately. all all openssl Spectre Imagine how another profession may value this type of instant, infinitely-retained, zero-effort-to-record recall that translates into an immediately-usable action. A lawyer submitting a particular type of legal document? A teacher scoring a set of tests? You can probably think of more, but my point is that the concept of shell history at the intersection of white-collar software engineering work holds tremendous potential. Here’s an actual example taken from my own work. Your use case may be different, but one situation that I run into for a host that I’ve brought down for something like a kernel update. The feedback loop looks kind of like this: sometimes is unsealing a Vault instance Ah, I gotta unseal this Vault instance. Where was my unseal key again? Oh yeah, it’s in Bitwarden. I can grab it with : rbw rbw get 'Vault Unseal Key' Crap. What instance needs to be unsealed? I guess I can ask Consul: http :8500/v1/health/state/critical Oh. Can I get those nodes as a list of endpoints that I can use with something else? http :8500/v1/health/state/critical \
    | jq -r '.[].ServiceID' \
    | awk -F: '{ print "http://" $2 ":" $3; }' : This is a -example, because I don’t know that pipeline by heart. I start with the command, then slap on the command then filter iteratively, running the command between added pipes to see what I get in-between. Note sub httpie jq awk Okay. I’ve got my unseal key and the endpoint(s) that needs to be unsealed. I can glue them together with a loop: zsh http :8500/v1/health/state/critical \
  | jq -r '.[].ServiceID' \
  | awk -F: '{ print "http://" $2 ":" $3; }' \
  | while read -r h
      do
        VAULT_HOST=$h vault operator unseal $(rbw get 'Vault Unseal Key')
      done The final command here is brusque and sort of obtuse. It’s not hard to see how you could clean this up and place it into a neatly-codified shell function or script: a variable for my Vault unseal key, a function to extract a list of sealed hosts, and so on. But I do this kind of short-scripting a . And the “amount of little pipelines I write the time it takes to turn them into good shell scripts and I put into the right file and commit to my dotfiles repo” number would be non-trivial. So I just skip it! lot ⨉ really I used to judge myself for this. However, over the course of many years, the number of times that I’ve kicked myself for not recording my many pipelines in a script somewhere versus in my shell history is very small to non-existent. I did have one large regret: when I didn’t take them with me across my workstations. But did you know there are to help you carry around your shell history with you? That’s no longer a big problem! tools specifically designed So what happens when you wrap up large and sometimes-complicated actions within your shell history, do it often, and let your grow into a beautiful, terrifying garden? .shell_history You can get , and the overhead to both a) record those shortcuts and b) use them is essentially frictionless. With the power of something like at your fingertips, the becomes available at a moment’s notice. You aren’t constrained to what you felt was worth overcoming any barrier to recording it for posterity. Everything is there, and you didn’t have to think about documenting any of it. so much done fzf entire history of your professional career in command-line work As an example: I haven’t worked directly with low-level S3 metrics in a long time. But I recall needing to aggregate total traffic through an S3 bucket in the past. I can find the command to answer that question for you in about two seconds by typing : do Ctrl-r aws s3 statistics <RET> today="$(date +%s)"
for bucket in $(aws s3 ls | awk '{ print $NF; }')
do
  echo -n "$bucket "
  aws cloudwatch get-metric-statistics \
    --namespace AWS/S3 \
    --start-time "$(echo $today - 86400 | bc)" --end-time "$today" \
    --period 86400 \
    --statistics Average \
    --region us-east-1 \
    --metric-name BucketSizeBytes \
    --dimensions Name=BucketName,Value=$bucket Name=StorageType,Value=StandardStorage \
      | jq '.Datapoints[].Average' \
      | gnumfmt --to=si
done This probably can’t be used 100% as written for a similar task a few years later, but smash that , give it a quick edit, and move on. Not too hard, and you can see why committing this to something very formal like a shell function just isn’t worth the overhead. It was specific then, it needs to be adapted now, and using it in this context is a very good match for “take it from my history and tweak it”.¹ Ctrl-x Ctrl-e tl;dr Try this: Tweak your shell history setup to retain a of history. Maybe even setup something like to bring it along with you wherever you go (or at least use history integration to make your better) lot atuin fzf’s Ctrl-r Where you can, try expressing tasks or units of work within self-contained shell commands.²  This means longer and more strung-together shell commands, but that’s fine! Your goal is to gradually accumulate a library of small programs that can replace any actions of yours that may otherwise be manual. Your future self will thank present-day self when you can re-use a command that does something hard or complicated with a few keystrokes. Footnotes ¹ I’ll acknowledge that examples like break down a bit in a team situation. Although I can punch up our S3 traffic in one second, how do you disperse that knowledge and capability to others? Big repositories of docs and scripts can solve this, but my purpose in this post is to drive home the fact that low friction cost and low barrier-to-entry cost is what makes the history shine. Maybe there’s a good way to make that and team sharing work in tandem. ² There’s another post buried here about “command-line first work”, but suffice it to emphasize that I think there are great benefits to be had by consolidating as much work as you can into command-line form. Easily-recalled actions in the form of shell history are one benefit alongside many others like interoperability, repeatability, and more. Also published . here

Alongside

Glue

Faster, Stronger, and 100% More Custom: A Grand Tour of Linux Kernels

An Interview with a Recovering Site Reliability Engineer

Nominated for 2022 - HackerNoon Contributor of the Year - Freelancing

Nominated for 2022 - HackerNoon Contributor of the Year - Linux

Too Long; Didn't Read

Make resilience your competitive advantage

Poor Command-Line Hygiene? Consider Load-bearing the Shell History

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

20,000 Leagues Under Your Shell

20,000 Leagues Under Your Shell

47 Stories To Learn About Bash

A Basic Guide to MySQL Tests Automation

A Web Service Written in Pure Bash.

20,000 Leagues Under Your Shell

20,000 Leagues Under Your Shell

47 Stories To Learn About Bash

A Basic Guide to MySQL Tests Automation

A Web Service Written in Pure Bash.

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps