The Command Line

Published: Posted on

I’ll often be working on my laptop when a colleague walks past, sees a command-line window open with lines scrolling and says something like, “well that looks scary!”. The producers of the matrix captured this sentiment well with their now classic screen image that shows indecipherable characters cascading down a screen.

I think that one of the least appreciated tools might be the command line. Most folks assume that it is the exclusive domain of software engineers (or Matrix insurgents) and miss out on the efficient data manipulation that is just one step away on the command line (or if you write your own code, but we’ll save that for another post!).

I suspect that one of the reasons the command line is daunting for tool users is because it does not – as Heidegger frames it – disclose itself freely (Erschlossenheit is the way he puts it in Being and Time). An icon bar at the bottom of your screen presents a limited set of clearly presented, intelligible options, and each icon conveys something of what you can hope to achieve by clicking. The command line is completely open-ended, just a cursor waiting for input. This isn’t strictly true, but it is fair to say that the user needs to invest a bit of time discovering what lies inside this black box. There are many rewards to those who peer under the lid, however, as I’ll detail in many of my posts on this blog. Command line tools can process large files and data sets very quickly (especially if you’re working with plain-text!). If you want to get data from the internet there are few more efficient ways than using cURL (learn more here). Want to find a specific pattern within a long text file or data set? Grep is amazing for finding any sort of text, and if you want to find something that fits a particular pattern – like say all the postcodes in a given dataset – you can learn about regular expressions (great examples on ProfHacker here and a quick reference from Microsoft). I’ll be diving into a number of examples of ways to use the command line in later posts, but today, I’ll  highlight one particular tool that I find indispensable: homebrew.

While many of the commands you’ll use are included as part of the default installation of MacOS (which is after all, just a pretty GUI overlaid on top of FreeBSD), there are a number of others which you will need to install yourself. Homebrew takes what can be a fairly convoluted process and makes it elegantly simple to install new command line tools.

To install homebrew, assuming you’re using a Mac, just open up terminal (it’s in the Applications/Utilities folder, though I use iTerm2 as a replacement) and paste in the following command:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
As the developers of homebrew put it, “Homebrew installs the stuff you need that Apple didn’t”. However, it is worth noting that this command will probably trigger your mac to install the XCode command line tools, which basically just includes a few basic tools to get you started.
I use about 2-3 dozen tools installed via homebrew, to manage bibtex files (bib-tool), to tweak csv databases (csv-fix), to run my own database server locally (mysql, postgresql and postgis), to convert multimarkdown into other formats (pandoc and pandoc-citeproc), to fix scanned PDF files (scantailor) to run encrypted filesystems inside dropbox (encfs… because Condoleza Rice is on the board of directors of DropBox folks), to edit file metadata (exiftool and tag). There are many others, but I’ve already belaboured the point. To install a tool using homebrew the process is simple. To search for a homebrew “cask” (install package) for a tool by name just type (substitute your search for “NAME”):
brew search NAME
and to install that tool, just type:
brew install NAME
Keep reading for more explanations on how I use these tools in my daily work processing data, writing journal articles, and sorting out bibliographies. Or just dive in and find some tools of your own! You can also check out a series that came up on the ProfHacker blog a few years ago: “The ProfHacker Guide to the Command Line” for some good ideas on how to get started.
So what command line tools do you rely on? Any favourites out there?

Leave a Reply

Your email address will not be published.