Perl Hacks

Hack #26. Iterate and Generate Expensive Data

Hide lists, streams, and expensive data structures behind a simple interface.

Perl's fundamental aggregate data types—hashes and arrays—are wonderfully flexible and often just what you want. That's often, not always. Sometimes you really need to process data that's expensive to calculate, part of a huge list that won't fit into memory, or just never ends.

When that happens, use a function reference as a data structure. Seriously.

The Hack

Imagine that you've just taken a job as a network administrator, replacing someone who completely failed to do any documentation. You know that you have all sorts of devices on the network with static IP addresses and you have a rough idea of the network blocks, but you don't know which addresses are in use.

Rather than finding every device, checking its settings, and reassigning things, you can write a little program to loop through each address and try to contact the device. It's a good first approximation. How do you check every netblock though? Use Net::Netmask to generate a list of IP addresses.

That could get messy though—do you really want to loop over a list of potentially millions of addresses? This is a good place to use a generator.

use Net::Netmask; sub create_generator { my @netmasks; for my $block (@_) { push @netmasks, Net::Netmask->new( $block ); } my $nth = 1; return sub { return unless @netmasks; my $next_ip = $netmasks[0]->nth( $nth++ ); if ( $next_ip eq $netmasks[0]->last( ) ) { shift ...