Learning Unix for OS X

Chapter 1. Why Use Unix?

Why would any sane person want to type in a bunch of funny-looking Unix commands when you can just use the trackpad? After all, OS X has one of—if not the—best looking user interfaces out there, so what would compel you, a Mac user through and through, to use the Unix command line? That’s a tough sell, but you can boil it down to just one word: power.

Lying underneath the OS X interface is a powerful Unix system, ready to leap into action at a moment’s notice. All you have to do is command Unix to take action. One of the greatest pleasures of using Unix within OS X is that you get the benefit of a truly wonderful graphical environment and the underlying power of the Unix command line. There’s no denying it’s a match made in heaven. Even Apple promotes OS X with the tagline, “The world’s most advanced desktop operating system.”

This chapter sets the stage for the rest of the book, answering the question: “Why use Unix when you have a perfectly good Mac graphical interface?” It’s an important question, but I think that if you give it a try, you’ll agree that joining the Unix world is really like learning you have a completely separate, and even more powerful, operating system lurking in your machine.

The Power of Unix

It’s quite reasonable to question why you should have to remember commands and type them in. If you’re a long-time Mac user who is familiar and happy with the capabilities and logic of the Aqua interface, you might need some convincing that Unix is your friend. Here’s why: dipping into the primarily text-based Unix tools on your OS X system gives you more power and control over both your computer and your computing environment. There are other reasons too, including:

• There are thousands of open source and otherwise freely downloadable Unix-based applications. Can’t afford Adobe Photoshop but still want a powerful graphics editor? The GNU Image Manipulation Program (GIMP) offers a viable solution (see Chapter 9).

• Want to search for files by when they were created, or by whom? Difficult in the Finder or Spotlight, but it’s a breeze with Unix (see Chapter 5).

• How about managing your files and file archives in an automated fashion? Tricky to set up with the GUI tools, but in Unix, you can set up a cron job to handle this at night while you sleep.

Fundamentally, Unix is all about power and control. As an example, consider the difference between using Force Quit from the Apple menu and the Unix programs ps and kill. While Force Quit is more attractive, as shown in Figure 1-1, notice that it lists only graphical applications.

Figure 1-1. Force Quit doesn’t show all running applications

By contrast, the ps (process status—say “pea-ess” to sound like a Unix guru) command used from within the Terminal application (/Applications/Utilities/Terminal) shows a complete and full list of every application, utility, and system process running on your Mac, as shown here:

$ ps -acx
  PID TTY           TIME CMD
    1 ??         0:10.28 launchd
   11 ??         0:01.41 UserEventAgent
   12 ??         0:01.86 kextd
   14 ??         0:01.15 notifyd
   15 ??         0:02.83 securityd
   16 ??         0:00.24 diskarbitrationd
 ...
 1526 ??         0:51.39 iTunes
 1573 ??         0:00.07 taskgated
 1583 ??         0:24.08 Google Chrome Helper
 1539 ttys000    0:00.04 login
 1540 ttys000    0:00.04 -bash
 1568 ttys000    0:00.21 vi
 1586 ttys001    0:00.03 login
 1587 ttys001    0:00.02 -bash

That’s more than the few applications Force Quit shows you. Of course, the next thing that’s probably running through your head is “Sure, but what does all that output in the Terminal mean to me, and what do I do with it?” This is the key reason to learn and work with the Unix side of OS X: to really know what your Mac’s doing and be able to make it match what you want and need your Mac to do.

Okay, now let’s go back and look at the output from running the ps -acx command. First off, you’ll see that we added some options (or flags or switches) to the ps command; the options are the -acx bit. Flags are spoken by letter, so this would be pronounced as “pea-ess minus aye-sea-ex.” These options tell ps to display all of the programs and processes being run by all of the users (including you and the system, itself) on the system. When the Terminal displays the results of the ps -acx command, you’ll see that it adds a line of “headers” to the output:

$ ps -acx
  PID TTY           TIME CMD
   16 ??         0:00.24 diskarbitrationd

Think of the headers the same way you would when looking at a Numbers spreadsheet with a bunch of columns. Each column in that spreadsheet should have a column head to help define what you see underneath. The same applies here. In the very first line of the information returned, you’ll see the following headers:

PID

Lists the command’s process identification number (or PID, for short).

TTY

Tells you the terminal the process is running in. If you see two question marks (??), that means the process isn’t associated with a specific Terminal window or display: typically it’s a system-level command or utility, as is the disk arbitration program listed above (diskarbitrationd—the final “d” stands for daemon, an always-running system-level task).

TIME

Tells you the amount of time it took to run that particular process, or how long that process has been running, in minutes and seconds. For example, the 0:00.24 you see in the output above means that it took, roughly, a quarter of a second for the diskarbitrationd process to start and run.

CMD

Gives you the specific command that’s being run. You can also ask for the entire pathname to the process that’s running, including any starting flags or options that might have been invoked. For example, /sbin/diskarbitrationd tells you that the process that’s running is diskarbitrationd, located in the /sbin directory.

Great! So now you know what all that means, but you still don’t know how this relates to Force Quit, right? Well, be patient, we’re getting there.

Once you know a process’s PID number, you can then issue the Unix kill command to, well, kill that process. For example, let’s say that Microsoft Word decides to lock up on you and you’re stuck with the Spinning Beach Ball of Death (SBBoD). After you finish tearing out your hair in frustration, you need to kill Microsoft Word, but in order to do so, you first need its process number. For this, we’ll add the grep command, which is basically a Unix search tool that you use to search for words or numbers in files, or in this case, the output of a command:

$ ps -ax | grep Word
 1634 ??         0:02.50 /Applications/Microsoft Office 2011/Microsoft
                 Word.app/Contents/MacOS/Microsoft Word -psn_0_766139
 1645 ttys002    0:00.00 grep Word

This tells us that Microsoft Word’s PID is 1634, as noted by the first number in the command output. Now all you need to do to kill Word is issue the following command:

$ kill 1634

After typing that and hitting the Return key (an activity known as “entering a command”), Microsoft Word promptly quits, closing all its windows. It won’t save anything you’ve done, since your last save, but since Word was locked in a deep freeze you wouldn’t have been able to save your changes anyway, right? And if you had used the Force Quit window, you wouldn’t have been able to save changes there, either.

Batch Renames and Extracting File Lists

Here’s another example. Suppose you just received a thumb drive from a client with hundreds of files in a single folder. Now let’s say that you only need those files that have the sequence -nt- or -dt- as part of their filenames, and that you want to copy them from the thumb drive to your home directory. Within the Finder, you’d be doomed to going through the list manually, a tedious and error-prone process. But on the Unix command line, this becomes a breeze:

$ cd /Volumes/MyCDROM
$ cp *-dt-* *-nt-* ~

The first command, cd /Volumes/Thumb, takes you to the Volumes directory, which is where the thumb drive (named Thumb) is actually mounted on your Mac’s filesystem. The second command, cp *-dt-* *-nt-* ~, breaks down as follows:

cp

This is Unix’s copy command.

*-dt-* *-nt-*

This tells the cp command to look for any items on the thumb drive that contain either -dt- or -nt- in their filenames. Unix recognizes the asterisks (*) as wildcards in the command string. By placing an asterisk before and after each item (*-dt-* and *-nt-*), you’re telling Unix to find any file that has either -dt- or -nt- anywhere in its filename.

~

The tilde character (or squiggle, in Unix-speak) simply refers to the current user’s home folder (or directory).

By placing the tilde (~) at the end of the command line, you’re telling cp to locate any file that has -dt- or -nt- in its filename, and copy those files to your home directory.

Fast, easy, and doable by any and all OS X users.

There are a million reasons why it’s helpful to know Unix as an OS X power user, and you’ll see them demonstrated time and again throughout this book.

Finding Hidden Files

You might not realize it if you only work in the Finder, but your system has thousands of additional files and directories that are hidden from view, but easily found on the command line. Most of these hidden files are known in the Unix world as dot files, because each file or directory has a period (.) as the first character of its name. For example, in your home directory you probably have a file called .profile that contains specific instructions on how you want your command shell set up when it’s launched. But when you view your home folder in the Finder, this file is hidden, as shown in Figure 1-2. Instead, all you see are the default set of folders (Desktop, Documents, Movies, Music, Pictures, Public, and Sites) and a file called myopen.

To view the dot files in the Terminal, type the file listing command (ls), along with its -a option (for list all, which shows the hidden dot files). Suddenly you’ll see that there are lots more files in that directory:

$ ls -a
./                   .dropbox/      .vuescanrc  Pictures/
../                  .dvdcss/       Desktop/    Presentations/
.CFUserTextEncoding  .lesshst       Documents/  Public/
.DS_Store            .nchsoftware/  Downloads/  Sites/
.Trash/              .profile       Dropbox/    bin/
.android/            .ssh/          Library/
.bash_history        .subversion/   Movies/
.cups/               .viminfo       Music/

Figure 1-2. The Finder doesn’t show hidden files and folders that you can see in the Terminal with standard Unix commands

Personally, though I don’t always need the power, I like knowing that I can get to, view, and even edit every file on my computer if I need to. All I need to do is launch the Terminal application (which I actually have permanently available on my Dock, so it’s always just one click away), type in a few simple commands, and I’m on my way.

Folders or Directories?

If you’re new to the whole Unix thing, you’re going to need to learn Unix-speak. In the graphical world, such as with OS X or Windows, you’re used to working with a graphical user interface (GUI) that lets you see everything visually. When you create a new file, it gets stored in a folder of some sort, even if you save the file to your Desktop (which is, in its own right, a folder).

But in Unix, folders are referred to as directories. That’s right, folders and directories are one and the same. It’s an odd sort of translation, but when Unix was first developed, there was no GUI; all you had was a text-based terminal to type into, and you were darn happy to have that, especially as you were walking 10 miles uphill in the snow to and from school each day. But I digress!

Directories were set up as part of the hard drive’s filesystem, or the structure in which directories and files are stored on the system. And the way you get to a folder (er, directory) in Unix is to enter its file path, using forward slashes between the directory names. For example, the file path to your home directory (again, think folder) is actually:

/Users/your_name

where your_name would be replaced by your short username. Or use the power user shortcut ~, as shown earlier!

At the very top of your Mac’s filesystem, you have the root directory, denoted with a single forward slash (/). As noted in the example above, to specify a particular directory or file all you need to do is place the path after this leading slash.

This takes a little getting used to, but once you get the hang of entering Unix file paths, you’ll find that it’s actually a faster way to get around (particularly if you can type faster than it takes you to move the cursor around in the graphical world).

Just remember: folders are directories, and when working on the Unix side of your Mac, we’ll refer to folders as directories throughout the book.

Thousands of Free Applications

This should appeal to anyone who is a part of the Macintosh community: by warming up to Unix and its command line, you are joining the much-lauded free software movement, since OS X is based on a free, open source Unix operating system called Darwin. What’s excellent is that there are thousands of different applications available for open source operating systems, including design, development, scientific, and business applications that compare quite favorably to expensive commercial alternatives. And don’t make the mistake of assuming that all open source applications are command-line tools and utilities! Some of the very best applications, like the GIMP graphics editor (http://www.gimp.org) and the NeoOffice suite (http://www.neooffice.org), are designed to work either within the X Window System (also known as X11), a standard Unix graphical interface that Apple includes with your OS X system, or directly in OS X’s Aqua GUI environment.

Power Internet Connections

If you’re someone who uses the Internet daily, you already know that there are a bunch of useful Mac OS X applications available to help you be more efficient. Unfortunately, lots of them seem to have a price tag attached—even a simple FTP program like Fetch (http://www.fetchsoftworks.com). But why spend $29 on an application when you can use OS X’s built-in ftp command-line utility for free?

For example, if you wanted to download the cover image for this book from O’Reilly’s website, you could use the following commands (as noted in bold type):

$ ftp ftp.oreilly.com
Connected to ftp.oreilly.com.
220 ProFTPD 1.3.1rc2 Server (ftp.oreilly.com) [208.201.239.51]
Name (ftp.oreilly.com:taylor): anonymous
331 Anonymous login ok, send your complete email address as your password
Password:
230-Welcome to the O'Reilly Media, Inc. FTP Archive.

 Local date and time: Mon Apr 23 23:04:14 2012

 --> Hello 71.237.2.63 <--
 --> There are 2 users out of 100 allowed in your usage class. <--

 Check us out on the web at http://oreilly.com
230 Anonymous access granted, restrictions apply
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd /pub/graphics/book-covers/low-res
250 CWD command successful
ftp> get 9781449332310.gif
local: 0596009151.gif remote: 0596009151.gif
229 Entering Extended Passive Mode (|||62244|)
150 Opening BINARY mode data connection for 0596009151.gif (267646 bytes)
100% |*******************************************************|   261 KiB
     430.20 KiB/s    00:00 ETA
226 Transfer complete
267646 bytes received in 00:00 (389.56 KiB/s)
ftp> bye
221 Goodbye.

That downloads the image file for the cover of this book to your Mac, which is nice, but what if you want to look at it? Sure, you could go to the Finder, find the file, and then double-click on the file’s icon to open it in Preview, but that’s a lot of work. Instead, with a little help from Unix, you can just type in the following command:

$ open 9781449332310.gif
$

The open command, which is special to OS X, examines the file it’s supposed to open (9781449332310.gif), detects which application should open it by default (something you can see in a file’s Get Info window), and then opens the file in Preview—all in a fraction of a second! See how much time Unix just saved you (not to mention the $29!)?

From logging into your Mac from remote locations to transferring files from your system to a server using an encrypted connection, OS X’s Unix command line is quite powerful. But don’t take my word for it—Chapter 8 takes you on a detailed tour of Internet command-line utilities.

Commands Included with Unix

While this book covers only about 50 of the most basic Unix commands, there are over a thousand Unix commands included with OS X—and you can’t see most of them without accessing the command line. From sophisticated software development environments to web browsers, file transfer utilities to encryption and compression utilities, almost everything you can do in the Aqua interface—and more—can be done with a few carefully chosen Unix commands.

Displaying All Unix Commands

To quickly see all of the binary executables—Unix programs—on your system, open the Terminal, hold down the Shift key, and press Esc-?, or press Control-X followed by Shift-1 (using Shift-1 to get an exclamation mark).

Before the commands are displayed in the Terminal, however, you’ll first be prompted (asked) to make a choice:

$
Display all 1663 possibilities? (y or n)

If you press the n key on your keyboard, you’ll be taken back to a command prompt and nothing else will happen. However, if you press the y key, you’ll see a multicolumn list of Unix commands stream past in the Terminal window. At the bottom of the screen, you’ll see:

--More--

This lets you know that there’s more to display. If you hit the space bar, the next “page” of commands scrolls into view. Keep pressing the space bar to view the entire list of commands; or, if you’re getting tired of that, just hit “q” to quit the output and go back to the command prompt.

If you’re a software developer or are just curious about programming, for example, you’ll want to install the optional Xcode Tools, available as a free download from the Apple Developer’s Site. The Xcode Tools give you a full, professional-grade software development environment that lets you develop new applications in Objective-C, C, or C++. Pretty nice for something free from Apple, eh?

The 10 Most Common Unix Commands

If you want to just jump in and try things out, here are the 10 most common commands, with a very short summary of what each does:

ls

Lists files or directories.

cp original_file copied_file

Copies the original_file (or files) from one location to another.

mv original_file new_file

Moves a file or files; the original is deleted once the operation is complete.

rm filename

Removes a file, set of files, or folder(s) full of files.

Warning

Use the rm command with caution; there’s no “Trash” to which things are moved. Once you’ve used rm to delete something, it’s gone forever.

pwd

Displays your present working directory; this is where you currently are in the filesystem.

cd directory_name

Changes to the specified directory in the filesystem. Without any arguments, it’s a shortcut for changing back to your home directory.

man command_name

Accesses OS X’s built-in documentation for the Unix commands. To read the man page for the ls command, for example, type in man ls.

less filename

Displays a long text file, one screen at a time. Pressing the space bar gets the next page when you’re ready, and pressing Q at any time quits the program and returns you to the command prompt.

grep pattern filename(s)

Searches for the specified pattern across as many files as you desire—a fast way to find that email message you sent to Uncle Linder, for example.

top

Shows you which applications and processes are running on your system, including those that the Finder’s Force Quit window ordinarily hides.

A Simple Guided (Unix) Tour

Enough talking about what Unix can do; it’s time to flex your fingers, open up your Mac, and try a few commands so you can get a sense of how it all works!

The first step is to launch the Terminal application, through which you’ll interact with the command shell. Terminal is tucked into the Utilities folder within your Applications folder.

Since you’ll be using the Terminal application throughout this book (and hopefully in the future, as you grow more comfortable with Unix), you should drag the Terminal’s icon to the Dock so it’s always at the ready. Or, if the Terminal’s already running, you can Control-click on its icon in the Dock and select Options→Keep in Dock, as shown in Figure 1-3.

Figure 1-3. Control-click the Terminal’s Dock icon, and select “Keep in Dock” from the Options menu so it will always be there when you need it

Throughout the following examples, type in the commands you see in bold, pressing the Return key after each one (again, this is known as “entering a command” in Unix-speak). Preceding each command, I’ve included some comments to let you know what you’re about to do.

Without any arguments, the cd command moves you to your home directory:

$ cd

The pwd (present working directory) command shows you the path for the directory you’re currently in:

$ pwd
/Users/taylor

Use the ls command to list the files in your home directory; compare this listing with the picture of the Finder window shown in Figure 1-2. If you omit the -a option, all the hidden dot files stay hidden in this directory:

$ ls
Desktop      Dropbox     Music           Public
Documents    Library     Pictures        Sites
Downloads    Movies      Presentations   bin

Now let’s change directories to your Library folder:

$ cd Library

Use the ls command again to see what’s inside (there’s very little here you’ll need to mess with):

$ ls
Accounts               FontCollections    PreferencePanes
Address Book Plug-Ins  Fonts              Preferences
Application Support    Fonts Disabled     Printers
Assistants             Frameworks         PubSub
Audio                  Google             Safari
Automator              Icons              Saved Application State
Autosave Information   Input Methods      Screen Savers
Breakpad               Internet Plug-Ins  Services
Caches                 Keyboard Layouts   Snapz Pro X
Calendars              Keychains          Sounds
ColorPickers           LaunchAgents       Spelling
Colors                 Logs               StickiesDatabase
Compositions           Mail               SyncedPreferences
Containers             Mail Downloads     Voices
Cookies                Messages           WebKit
Developer              Metadata           Widgets
FIPLAB                 Mobile Documents   disclabel
Favorites              PDF Services       iMovie
FileSync               Parallels          iTunes

Now let’s go back a directory. For this, use the .. shortcut for moving up one directory in the filesystem. In this case, since you were in your Library folder (e.g., /Users/taylor/Library, or just ~/Library), the following command moves you back to your home directory (as noted by the pwd command that follows):

$ cd ..
$ pwd
/Users/taylor

Finally, when it’s time to quit the Terminal, use the exit command rather than just quitting the application with ⌘-Q:

$ exit

Don’t worry if you aren’t sure exactly what each of those commands does: we’ll explore each one in great detail as the book proceeds.

There’s a whole world of Unix inside your OS X system, and it’s time for you to jump in and learn how to be more productive and more efficient, and gain remarkable power as a Mac user. Ready? Let’s go!