Hackernoon logoYSK about [Unix tools] by@austin

YSK about [Unix tools]

Austin Pocus Hacker Noon profile picture

@austinAustin Pocus

Do not attempt to adjust your set. We are controlling transmission.

This Slack discussion, featuring myself in some sleep-deprived rambling, occurred in hackernoon's official #protips channel, and has been [barely] edited for readability.

austinOct 14, 2020, 2:30 PM
YSK about
lsof
, short for “list open files”. since everything, literally everything from network sockets to camera handles, is a file in unix-based systems (like macos). this means you can do things like

lsof | grep "AppleCamera"


in the terminal, to see if anyone’s using your camera (even if the little light is off)

or you can do

lsof -i -P -n | grep LISTEN


to see all programs with open network ports, if you want to say, kill a webserver that ran away on you, to open up the port. or if you’re locking down a machine. the thing’s a power tool
austinOct 14, 2020, 2:33 PM
or
netcat
! talk about power tools. that thing will send literally anything over anything. want to test an old PPP connection? send a “malicious” packet to test your webserver’s mettle? see what happens when you send a UDP formatted packet over TCP?
netcat
. more unix tools incoming, in this thread
austinOct 14, 2020, 2:35 PM
how about
tcpdump
? it’s basically wireshark before wireshark was cool — if you’re on an ssh connection in particular, might be worth brushing up on
tcpdump
(i know, wireshark can work over an ssh connection, but imagine you’re on a desert island. with a computer.) point is, these old school tools can be useful sometimes too (and can be piped together like any ol’ unix tool)
austinOct 14, 2020, 2:36 PM
ok so it’s decided: this is a thread about useful unix tools. please don’t post in here, people — i’m going to turn this into a post later
austinOct 14, 2020, 2:37 PM
screen
is a good one too. (i’m just thinking these up off the top of my head, so if i seem all over the place, it’s because i am). the
screen
tool will keep a program running even if your ssh connection fails! talk about useful
austinOct 14, 2020, 2:39 PM
oh!
strace
/
ptrace
! those things are amazing. you can basically poke around inside a running program. think of it like game genie for your code
austinOct 14, 2020, 2:40 PM
curl
. one of the most underrated tools on unix…you can test an whole ass webserver with
curl
. i mean, there are a thousand tools for “REST testing” these days, but
curl
is the og.
austinOct 14, 2020, 2:42 PM
grep
, again underrated and used to death. especially powerful with unix pipes. i mean, you can combine grep with literally any other tool in this list, except maybe
screen
and only because that wouldn’t really make sense…unless you’re grepping redirected output? idk. point is, all these tools can talk to each other using a standard format, one we all know: text!
austinOct 14, 2020, 2:44 PM
btw, at this point, it’s worth mentioning: half the cool unix tricks i’ve learned, i learned from https://twitter.com/b0rk aka https://jvns.ca/, creator of http://wizardzines.com. check it! she knows more about unix and linux systems than i can probably ever hope to, and she presents it in a really fun, digestible format.
austinOct 14, 2020, 2:46 PM
let’s see…
ssh
is actually more powerful than most people realize. you can set up “chains” of ssh connections, you can set up all sorts of crazy network configs….i mean, it does more than just remote terminal logins.
austinOct 14, 2020, 2:47 PM
fun fact:
emacs
(and
vim
too, i believe) have an interface for editing remote files on your local instance! so no more using default
vi
to edit files over ssh! 😄
austinOct 14, 2020, 2:50 PM
awk
! i forgot about
awk
, because everyone does. basically, if you have data that’s spit out in a columnar format like
ps
, or
lsof
, or [insert any unix tool on the planet], you can use a simple
awk
command to pull a single value out of a given column, and so much more. which i don’t know off the top of my head because the columnar data processing is all i’ve really needed. but in a pinch, you can even use it like
grep
austinOct 14, 2020, 2:53 PM
sed
…here be dragons, folks. short for “stream editor”, sed is like find and replace on some super steroid derived from the blood of all the disgraced baseball players in the 90s. anyway, sed can do stuff like find and replace every single instance of a regex match with another string or pattern (including regex capture groups!), in place. meaning it will edit your files with no recovery plan, no backup. that’s why i say: here be dragons. this is a power tool so powerful, you can cut a limb off. be careful.
austinOct 14, 2020, 2:54 PM
tar
is useful, but damned if i remember how to use it. there’s even https://xkcd.com/1168/. i mean you can create the unix equivalent of zip files, and decompress them as well. it’s useful, i just…i need google to use it.
austinOct 14, 2020, 2:56 PM
find
is a good one. i mean, if you ever need to find something and don’t have the luxury of macos’s finder features, this is your tool. it can find files with a pattern, using modification time, and so much more. combine with a unix pipe and
xargs
for extra fun!
austinOct 14, 2020, 2:56 PM
xargs
, that’s useful as well. basically it’s the unix equivalent of the “spread operator” in JS, if that makes sense.
austinOct 14, 2020, 2:59 PM
two more: the first,
man
, is essential for any unix or linux user. it’s quite simply, the manual pages. use it as a system reference, for remembering what a given flag does, or whenever
$TOOLNAME --help
doesn’t cut it
austinOct 14, 2020, 3:00 PM
the last is
apropos
— literally. this will, given a keyword, tell you which tools are appropriate for your use case! like if you type
apropos file
, it’ll tell you about file-related tools. it’s like the index they used before search engines sort of drove it into the shadows
austinOct 14, 2020, 3:01 PM
hope you enjoyed this sleep deprived rant! back to my regularly scheduled programming…

Tags

Join Hacker Noon

Create your free account to unlock your custom reading experience.