Console Into Mass. The Transit to the Light Side. Automatization of Routine Tasks

Introduction

In spite of what we do and how fast we enter the commands, the reality is, we still can’t beat the performance of computers. From the other hand, if we keep repeating the same action multiple times, we can easily give computers a hard time, can’t we? You could write a bash script (your favorite programming language) and instead of entering the same commands, wasting your time and energy, run the script and have some time for yourself, sit back in a seat, think of the eternity, universe, or anything else that comes up on your mind.

In the previous article we discussed the principles of bash programming. Today we will learn how to apply this knowledge in practice.

Automatization Plan

1. Fast diff
2. Fast diff + Jira API
3. Clean _dist
4. Updating a large number of repositories
5. Cloning a large number of repositories
6. Useful aliases

The given plan comprises some of the tasks that I do several times a day (sometimes even an hour) every day. Generally speaking, automatization is a unique and original process that allows automating anything you can think of. Hopefully, by the time you’ll have read this article to the end, you will come up with your own automatization plan on how to surpass your PC performance. So, before we get started, make yourself a cup of hot coffee and enjoy our adventure to the world of automatization with bash.

Fast diff

Personally, I prefer to use Git. Creating diff is a frequent task that requires entering the following command:git diff origin/master origin/<branch-name> > "${HOME}/diff/diff-<branch-name>.diff"

<branch-name> — is the name of the branch for which we need to create diff

The disadvantages of the given approach

1. Entering the command is done only manually
2. Risks to mistype the command
3. Not easy to remember

If you use bash, however, these issues can be solved. You should be able to:

1. Enter the command
2. Enter the name of the branch
3. Get diff

And do all this without a hitch.

This is how the end command looks

gdd <branch-name>

Automatize

Now, instead of typing a long command, it’s enough to enter ./fast_diff.sh <branch-name>. The script will remind you of inputting the name of the branch in case you forget to do it.

The final touch

Coming at this point you may wonder about the end command since using the script as it isn’t exactly convenient, given that we’re still bound to the directory we use.

Let’s take a closer look how it’s possible to create a new command for the executive file, not writing a relative/absolute path to it every time.

Every user has a subdirectory ~/bin that stores executive files. If you don’t, you can easily create it. What makes its use so convenient is that all files there can be accessed by name and there’s no need to specify their paths. To this subdirectory I’ve moved gdd file used to create diff:#!/bin/bash  "${HOME}/htdocs/rybka/tools/fast_diff.sh" "$@"

Some important notes:

1. There’s no need to specify a file extension.
2. The attribute x should be specified in a clear way (chmod +x <filename>)
3. If you don’t find bin in $PATH variable, you should make it more apparent by typing: PATH="${PATH}:${HOME}/bin".

Relaunch the terminal to make this file accessible. Now, in order to create diff, you only need to enter the given command:

gdd <branch-name>

If you don’t like an idea of creating a new file for each command, you can optimize this process using a symbolic link:

ln -s "${HOME}/htdocs/rybka/tools/fast_diff.sh" gdd

Fast diff + Jira API

If you use Jira or any other task manager with API, you can go even further. With the help of Jira API, for example, the diff can be assigned to a certain task. To do so, you will need to use cURL.

Solution algorithm

1. Run the script
2. Define task id
3. If the task id hasn’t been provided, send user a message
4. Given that all has been done correctly, we generate diff and assign it to a task

This is how the end command looks

gdd_jira <issue_id>

Automatize

As you’ve probably noticed, this time we don’t need to pass the name of the branch to the script. We can get it by performing a few simple manipulations with the git commands:branch=$(git rev-parse — abbrev-ref HEAD)

Clean _dist

Before we go any further let’s see what _dist directory is used for. Simply said, it’s the place where all CSS, JavaScript, templates (Jade/Pug, Handlebars, etc) and other files are saved after the launch of build system (Grunt, Gulp, etc). The directory doesn’t necessarily have to bear the _dist name. You can find many different variations.

For one of the projects we use Grunt. Quite often, though, our team have encountered with a problem that it doesn’t always seem to see a change in some of the files, Less files in the majority. To solve this problem, though, you can clean the _dist directory for one or for all themes at once. To do this, you could also use Grunt, cleaning the directory manually, yet, it wouldn’t be same effective as it is with bash. There are much more directories here, not one or two, not even twenty. There are a lot of them. The main requirement for when working with the script is not to overload it with wrappers and/or dependencies without need.

Let’s see how the same can be done with the power of shell:find <path-to-themes> -type d -name "_dist" | xargs rm -rfv

<path-to-themes> — the path to the directory with the themes

The disadvantages of the given approach are the same as in the case of the diff creation. Plus, there’s no option to specify one theme that we’d like to delete _dist directory from.

Solution algorithm

1. Run the script
2. If you don’t have the name of the theme, you can delete _dist directory for all of them
3. If you have a certain theme name, delete _dist for only this one

This is how the end command looks

clean_dist [<theme_name>]

Automatize

Updating a large number of repositories

Imagine you work on a big project that contains a directory for third-party repositories which, even though you don’t create them, you still need to support and update. If they are two or three, that might not be a big problem, although, personally I’d not be so confident.

But what if you’ve got more than 10–15 repositories to support?

You’d need to spend a lot of time tracking them constantly. Why not automatize this process then?

Solution algorithm

1. Go to the directory with a repository
2. Check whether a repository is in the master branch
3. If it’s not there, run git checkout
4. Then git pull

An important note. Even if a repository has switched to master branch, there’s still a chance it might be not updated. With that in mind, running git pull ought to be done anyway.

This is how the end command looks

up_repo

Automatize

Cloning a large number of repositories

This process is closely connected to the previous one. To make it possible for a user to use a previous command in practice. I need to provide him/her with a repository of third-party developers adding it into a bash/core/vendors directory (a user doesn’t necessarily need to be aware of it). On the analogy of npm modules, this repository shouldn’t go alongside the main one. All a user should do is execute the command and wait until after the cloning of all repositories is completed.

Solution algorithm

1. The list of repositories is set as an array
2. Run the array cycle
3. Pay a particular attention if a vendor has more than one repository
4. Run a few more additional checkups
5. Execute git clone

This is how the end command looks

clone_repo

Automatize

Useful aliases

I’d like to ask my reader a couple of questions and I’d like you to answer them honestly. How often do you use this command?git branch

What about this command?git status

And this one?git push origin <branch-name>

Do you often use this one?ps aux | grep <user-name>

This list can be extended and more likely than not, everyone has got his/her frequently used commands. So, here is where you may suddenly think that for all these commands it’d be a sensible idea to create aliases.

In the list below you’ll find some of the aliases that I use on a daily basis:

In order to check what aliases have been set, you want to run the command alias but without specific parameters.

Where to save aliases

To create a permanent alias, add it into .bashrc file in a user’s home (~) directory. You could also add it into .gitconfig file for working with git.

Don’t change aliases late at night

Alias is a powerful tool. Yet, here is all as with passwords.

Don’t change your aliases when you’re about to go to bed.

I did happen to change one late at night. What happened next you can guess. I didn’t remember I’d done it and I spent most part of a day trying to figure out why nothing worked.

In conclusion

When I just got into the principles of bash, my first thought was: “Stop, aren’t they what system administrators need?”. Yet, I did understand the importance of this knowledge that would allow me to lighten the burden of daily routine tasks. Today, I can say with confidence that if your work is anyhow related to remote servers or OS *nix,or you work with Windows OS (Bash on Ubuntu on Windows, Windows and Ubuntu Interoperability), the knowledge and understanding of the principles of bash will come absolutely useful.

Simply put, a script is nothing more than a basic list of system commands, written all in one file.

Using this file, though, can simplify executing a lot of routine tasks that otherwise, you’d do manually.

Here are a few links with bash capabilities, some of which I’ve shown in my examples:

1. I/O Redirection
2. Functions
3. Arrays
4. The Double-Parentheses Construct
5. Special Characters (pipe)
6. Exit and Exit Status
7. Aliases
8. How to add paths to $PATH variable

That’s it. Thanks for your attention and special thanks to those who’ve read the article to the end.