Exploring Expect

Handling Timeout

Much of the time, expect commands have only one argument—a pattern with no action—similar to the very first one in this chapter:

expect "hi"

All this does is wait for hi before continuing. You could also write this as:

expect "hi" {}

to show the empty action, but expect does not require it. Only the last action in an expect command can be omitted:

expect  {
    "hi"            {send "You said hi\n"}
    "hello"            {send "Hello yourself\n"}
    "bye"
}

As a natural consequence of this, it is typical to write expect commands with the exception strings at the top and the likely string at the bottom. For example, you could add some error checking to the beginning of the anonymous ftp script from the previous chapter:

spawn ftp $argv
set timeout 10
expect {
    "connection refused" exit
    "unknown host" exit
    "Name"
}
send "anonymous\r"

If the script sees Name it will go on and send anonymous\r. But if it sees "unknown host" or "connection refused“, the script will exit. Scripts written this way flow gracefully from top to bottom.

If, after 10 seconds, none of these patterns have been seen, expect will timeout and the next command in the script will be executed. I used this behavior in constructing the timed_read script in the previous chapter. Here, however, I only want to go to the next command if Name is successfully matched.

You can distinguish the successful case from the timeout by associating an action with the timeout. This is done by using the special pattern timeout. It looks like this:

expect {
    timeout {puts "timed out"; exit}
    "connection refused" exit
    "unknown host" exit
    "Name"
}

If none of the patterns match after ten seconds, the script will print "timed out" and exit. The result is that the script is more robust. It will only go on if it has been prompted to. And it cannot hang forever. You control how long it waits.

Although the timeout pattern is invaluable, it is not a replacement for all error handling. It is tempting to remove the patterns "connection refused" and "unknown host“:

expect {
    timeout exit
    "Name"
}

Now suppose "unknown host" is seen. It does not match Name and nothing else arrives within the ten seconds. At the end of ten seconds, the command times out. While the script still works, it fails very slowly.

This is a common dilemma. By explicitly specifying all the possible errors, a script can handle them more quickly. But that takes work on your part while writing the script. And sometimes it is impossible to find out all the error messages that a program could produce.

In practice, it suffices to catch the common errors, and let timeout handle the obscure conditions. It is often possible to find a pattern with appropriate wildcards that match many errors. For example, once ftp is connected, it is always possible to distinguish errors. ftp prefaces all output with a three-digit number. If it begins with a 4 or 5, it is an error. Assuming ftp’s line is the only thing in expect’s input buffer, you can match errors using the range construct described on page 91:

expect {
    timeout {unexpected ...}
    "^\[45]" {error ...}
    "ftp>"
}

As I described in Chapter 3 (p. 73), the ^ serves to anchor the 4 or 5 to the beginning of the buffer. If there are previous lines in the buffer—as is more likely—you can use the pattern "\n\[45]“. The linefeed (\n) matches the end of the carriage-return linefeed combination that appears at the end of any line intended to be output on a terminal.

When the timeout pattern is matched, the data that has arrived is not moved to expect_out(buffer). (In Chapter 11 (p. 248), I will describe the rationale for this behavior.) If you need the data, you must match it with a pattern. You can use the * wildcard to do so:

expect *

As I noted earlier, this command is guaranteed to return immediately, and expect_out(buffer) will contain what had arrived when the previous timeout occurred.

By convention, the timeout pattern itself is not quoted. This serves as a reminder to the reader that expect is not waiting for the literal string "timeout“. Putting quotes around it does not change expect’s treatment of it. It is still be interpreted as a special pattern. Quotes only protect strings from being broken up, such as by spaces. For that reason, you can actually write a subset of expect patterns without any quotes. Look at the following intentionally obfuscated examples:

expect "hi" there
expect  hi  there
expect "hi  there"

In the first and second commands, hi is the pattern, while "hi there" is the pattern in the third command. For consistency, use quotes around all textual patterns, and leave them off the special pattern timeout. In Chapter 5 (p. 109), I will show how to wait for the literal string timeout.

Here is another example of the timeout pattern. You can use the ping command to test whether a host is up or not. Assume that host elvis is up and houdini is down. Not all versions of ping produce the same output, but here is how it looks when I run it:

% ping elvis
elvis is alive
% ping houdini
no answer from houdini

What ping actually does is to send a message to the host which the host should acknowledge. ping usually reports very quickly that the host is up, but it only says "no answer" after waiting quite a while—20 seconds is common.

If the host is on your local network, chances are that if the host does not respond within a second or two, it is not going to respond. If you are only looking for a host to farm out some background task, this heuristic works well. Realistically, it is exactly the same heuristic that ping uses—just a little less tolerant. Here is an Expect script that forces ping to respond after 2 seconds.

spawn ping $host
set timeout 2
expect "alive" {exit 0} timeout {exit 1}

If the expect sees alive within two seconds, it returns 0 to the caller; otherwise it returns 1. When called from a /bin/sh script, you find the result by inspecting the status. This is stored in the shell variable $? (or $status in csh).

$ echo $?
0

Strictly speaking, the status must be an integer. This is good in many cases—integers are easier than strings to check anyway. However, it is possible to get the effect of returning a string simply by printing it out. Consider the following commands which print out the same messages as ping:

spawn ping $host
set timeout 2
expect "alive" {exit 0} timeout {
    puts "no answer from $host"
    exit 1
}

The timeout action prints the string "no answer from ..." because the script will abort ping before it gets a chance to print its own error message. The alive action does not have to do anything extra because ping already prints the string. Both strings are sent to the standard output. In Chapter 7 (p. 171), you will see how to prevent printing strings from the underlying process, and even substitute your own if desired.

Some versions of ping have a user-settable timeout. But the technique I have shown is still useful. Many other programs are completely inflexible, having long fixed timeouts or none at all.

rsh is a program for executing shell commands remotely.^[19]rsh is an example of a program that is very inflexible when it comes to timeouts. rsh waits for 75 seconds before deciding that a machine is down. And there is no way to change this time period. If rsh finds the machine is up, it will then execute the command but without any ability to timeout at all. It would be nice if rsh and other commands all had the ability to timeout, but it is not necessary since you can achieve the same result with an Expect script.

Rather than writing separate scripts to control rsh and every other problem utility, you can write a parameterized script to timeout any program. The two parameters of interest are the program name and the timeout period. These can be passed as the first and second arguments. Assuming the script is called maxtime, it could be used from the shell to run a program prog for at most 20 seconds with the following:

% maxtime 20 prog

Here is the script:

#!/usr/local/bin/expect --
set timeout [lindex $argv 0]
spawn [lindex $argv 1]
expect

The script starts by setting the timeout from the first argument. Then the program named by the second argument is spawned. Finally, expect waits for output. Since there are no patterns specified, expect never matches using any of the output. And because there are no patterns to match, after enough time, expect times out. Because there is no timeout action, expect simply returns, and the script ends. Alternatively, if the program ends before the timeout, expect notices this and returns immediately. Again, the script ends.

---

^[19] Some systems call it remsh.