Make sure that your process stays up, no matter what
$ while : ; do echo "Run some code here..."; sleep 1; done
If you run a foreground process in place of that
echo line, then the process is always guaranteed
to be running (or, at least, it will try to
: simply makes the
while always execute (and is more efficient than
running /bin/true, as it
doesn't have to spawn an external command on each
iteration). Definitely do not run a background
process in place of the echo, unless you enjoy
filling up your process table (as the while will
then spawn your command as many times as it can, one every second).
But as far as cool hacks go, the while approach
is fairly lacking in functionality.
What happens if your command runs into an abnormal condition? If it exits immediately, then it will retry every second, without giving any indication that there is a problem (unless the process has its own logging system or uses syslog). It might make sense to have something watch the process, and stop trying to respawn it if it returns too quickly after a few tries.
There is a utility already present on every Linux system that will do this automatically for you: init . The same program that brings up the system and sets up your terminals is perfectly suited for making sure that programs are always running. In fact, that is its primary job.
The inittab line consists of an arbitrary (but
unique) two character identification string (in this case,
zz), followed by the runlevels that this program
should be run in, then the
respawn keyword, and
finally the full path to the command. In the above example, as long
my_daemon is configured to run in the
foreground, init will respawn another copy
whenever it exits. After making changes to
inittab, be sure to send a HUP to
init so it will reload its configuration. One
quick way to do this is:
# kill -HUP 1
If the command respawns too quickly, then init will postpone execution for a while, to keep it from tying up too many resources. For example:
This will cause the file /tmp/timestamp to be touched several times a second, until init decides that enough is enough. You should see this message in /var/log/messages almost immediately:
Sep 8 11:28:23 catlin init: Id "zz" respawning too fast: disabled for 5 minutes
In five minutes, init will try to run the command again, and if it is still respawning too quickly, it will disable it again.
Obviously, this method is fine for commands that need to run as root, but what if you want your auto-respawning process to run as some other user? That's no problem: use sudo:
zz:12345:respawn:/usr/bin/sudo -u rob /bin/touch /tmp/timestamp
Now that touch will run as rob, not as root. If you're trying these commands as you read this, be sure to remove the existing /tmp/timestamp before trying this sudo line. After sending a HUP to init, take a look at the timestamp file:
rob@catlin:~# ls -al /tmp/timestamp -rw-r--r-- 1 rob users 0 Sep 8 11:28 /tmp/timestamp
The two drawbacks to using init to run arbitrary daemons are that you need to comment out the line in inittab if you need to bring the daemon down (since it will just respawn if you kill it) and that only root can add entries to inittab. But for keeping a process running that simply must stay up no matter what, init does a great job.