Perl Cookbook

Picking a Random Line from a File

Problem

You want to return a random line from a file.

Solution

Use rand and $. (the current line number) to decide which line to print:

srand;
rand($.) < 1 && ($line = $_) while <>;
# $line is the random line

Discussion

This is a beautiful example of a solution that may not be obvious. We read every line in the file but don’t have to store them all in memory. This is great for large files. Each line has a 1 in N (where N is the number of lines read so far) chance of being selected.

Here’s a replacement for fortune using this algorithm:

$/ = "%%\n";
@ARGV = ('/usr/share/games/fortunes') unless @ARGV;
srand;
rand($.) < 1 && ($adage = $_) while <>;
print $adage;

If you know line offsets (for instance, you’ve created an index) and the number of lines, you can randomly select a line and jump to its offset in the file, but you usually don’t have such an index.

Here’s a more rigorous explanation of how the algorithm works. The function call rand ($.) picks a random number between 0 and the current line number. Therefore, you have a one in N chance, that is, 1/N, of keeping the Nth line. Therefore you’ve a 100% chance of keeping the first line, a 50% chance of keeping the second, a 33% chance of keeping the third, and so on. The question is whether this is fair for all N, where N is any positive integer.

First, some concrete examples, then abstract ones.

Obviously, a file with one line (N=1) is fair: you always keep the first line because 1/1 = 100%, making it ...