By Randal L. Schwartz
With  Tom Phoenix

Cover | Table of Contents | Colophon

Table of Contents

Chapter 1: Introduction

Content preview·Buy reprint rights for this chapter

Welcome to next step in your understanding of Perl. You're probably here either because you want to learn to write programs that are more than 100 lines long or because your boss has told you to do so.

See, our Learning Perl book was great because it introduced the use of Perl for short and medium programs (which is most of the programming done in Perl, we've observed). But, to avoid having "the Llama book" be as big and intimidating as "the Camel book," we left a lot of information out, deliberately and carefully.

In the pages that follow, you can get "the rest of the story" in the same style as our friendly Llama book. It covers what you need to write programs that are 100 to 10,000 lines long.

For example, you'll learn how to work with multiple programmers on the same project. This is great, because unless you work 35 hours each day, you'll need some help with larger tasks. You'll also need to ensure that your code all fits with the other code as it is developed for the final application.

This book will also show you how to deal with larger and more complex data structures, such as what we might casually call a "hash of hashes" or an "array of arrays of hashes of arrays."

And then there's the buzzworthy notion of object-oriented programming, which allows parts of your code (or hopefully code from others) to be reused with minor or major variations within the same program. The book will cover that as well, even if you've never seen objects before.

An important aspect of working in teams is having a release cycle and tests for unit and integration testing. You'll learn the basics of packaging your code as a distribution and providing unit tests for that distribution, both for development and for verifying that your code works in the ultimate end environment.

Hopefully, just as was promised and delivered in Learning Perl, you'll be entertained along the way by interesting examples and bad puns. (We've sent Fred and Barney and Betty and Wilma home, though. A new cast of characters will take the starring roles.)

We'll presume that you've already read Learning Perl

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What Should You Know Already?

Content preview·Buy reprint rights for this chapter

We'll presume that you've already read Learning Perl, or at least pretend you have, and that you've played enough with Perl to already have those basics down. For example, you won't see an explanation in this book that shows how to access the elements of an array or return a value from a subroutine.

Make sure you know the following things:

• How to run a Perl program on your system
• Scalars
• Arrays
• Hashes
• Control structures such as while, if, for, and foreach
• Subroutines
• Perl operators such as grep, map, sort, and print
• File manipulation such as open, file reading, and -x (file tests)

You might pick up deeper insight into these topics in this book, but we're going to presume you know the basics.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What About All Those Footnotes?

Content preview·Buy reprint rights for this chapter

Like Learning Perl, this book relegates some of the more esoteric items out of the way for the first reading and places those items in footnotes. You should skip those the first time through and pick them up on a rereading. You will not find anything in a footnote that is needed to understand any of the later material.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What's with the Exercises?

Content preview·Buy reprint rights for this chapter

Hands-on training gets the job done better. The best way to provide this training is with a series of one or more exercises after every half-hour to hour of presentation. Of course, if you're a speed reader, the end of the chapter may come a bit sooner than a half hour. Slow down. Take a breather. But do the exercises.

Each exercise has a "minutes to complete" rating. This rating hits the midpoint of the bell curve but don't feel bad if you take significantly longer or shorter. Sometimes it's just a matter of how many times you've faced similar programming tasks in your studies or jobs. Use the numbers merely as a guideline.

Every exercise has its answer in Appendix A. Again, try not to peek; you'll ruin the value of the exercise.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What if I'm a Perl Course Instructor?

Content preview·Buy reprint rights for this chapter

If you're a Perl instructor who has decided to use this as your textbook, you should know that each set of exercises is short enough for most students to complete the whole set in 45 minutes to an hour, with a little time left over for a break. Some chapters' exercises should be quicker, and some may take longer. That's because once all those little numbers in square brackets were written, we discovered that we don't know how to add.

So let's get started. Class begins after you turn the page...

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 2: Building Larger Programs

Content preview·Buy reprint rights for this chapter

This chapter looks at how to break up a program into pieces and includes some of the concerns that arise when you put those pieces back together again, or when many people work together on the same program.

Let's say a famous sailor (we'll call him "the Skipper") uses Perl to help navigate his ocean-going vessel (call it "the Minnow"). The Skipper writes many Perl programs to provide navigation for all the common ports of call for the Minnow. He finds himself cutting and pasting a very common routine into each program:

sub turn_towards_heading {
  my $new_heading = shift;
  my $current_heading = current_heading(  );
  print "Current heading is ", $current_heading, ".\n";
  print "Come about to $new_heading ";
  my $direction = "right";
  my $turn = ($new_heading - $current_heading) % 360;
  if ($turn > 180) { # long way around
    $turn = 360 - $turn;
    $direction = "left";
  }
  print "by turning $direction $turn degrees.\n";
}

This routine gives the shortest turn to make from the current heading (returned by the subroutine current_heading( )) to a new heading (given as the first parameter to the subroutine).

The first line of this subroutine might have read instead:

my ($new_heading) = @_;

This is mostly a style call: in both cases, the first parameter ends up in $new_heading. However, in later chapters, you'll see that removing the items from @_ as they are identified does have some advantages. So, this book sticks (mostly) with the "shifting" style of argument parsing. Now back to the matter at hand...

Suppose that after having written a dozen programs using this routine, the Skipper realizes that the output is excessively chatty when he's already taken the time to steer the proper course (or perhaps simply started drifting in the proper direction). After all, if the current heading is 234 degrees and he needs to turn to 234 degrees, you see:

Current heading is 234.
Come about to 234 by turning right 0 degrees.

How annoying! The Skipper decides to fix this problem by checking for a zero turn value:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

The Cure for the Common Code

Content preview·Buy reprint rights for this chapter

Let's say a famous sailor (we'll call him "the Skipper") uses Perl to help navigate his ocean-going vessel (call it "the Minnow"). The Skipper writes many Perl programs to provide navigation for all the common ports of call for the Minnow. He finds himself cutting and pasting a very common routine into each program:

sub turn_towards_heading {
  my $new_heading = shift;
  my $current_heading = current_heading(  );
  print "Current heading is ", $current_heading, ".\n";
  print "Come about to $new_heading ";
  my $direction = "right";
  my $turn = ($new_heading - $current_heading) % 360;
  if ($turn > 180) { # long way around
    $turn = 360 - $turn;
    $direction = "left";
  }
  print "by turning $direction $turn degrees.\n";
}

This routine gives the shortest turn to make from the current heading (returned by the subroutine current_heading( )) to a new heading (given as the first parameter to the subroutine).

The first line of this subroutine might have read instead:

my ($new_heading) = @_;

This is mostly a style call: in both cases, the first parameter ends up in $new_heading. However, in later chapters, you'll see that removing the items from @_ as they are identified does have some advantages. So, this book sticks (mostly) with the "shifting" style of argument parsing. Now back to the matter at hand...

Suppose that after having written a dozen programs using this routine, the Skipper realizes that the output is excessively chatty when he's already taken the time to steer the proper course (or perhaps simply started drifting in the proper direction). After all, if the current heading is 234 degrees and he needs to turn to 234 degrees, you see:

Current heading is 234.
Come about to 234 by turning right 0 degrees.

How annoying! The Skipper decides to fix this problem by checking for a zero turn value:

sub turn_towards_heading {
  my $new_heading = shift;
  my $current_heading = current_heading(  );
  print "Current heading is ", $current_heading, ".\n";
  my $direction = "right";
  my $turn = ($new_heading - $current_heading) % 360;
  unless ($turn) {
    print "On course (good job!).\n";
    return;
  }
  print "Come about to $new_heading ";
  if ($turn > 180) { # long way around
    $turn = 360 - $turn;
    $direction = "left";
  }
  print "by turning $direction $turn degrees.\n";
}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Inserting Code with eval

Content preview·Buy reprint rights for this chapter

The Skipper can save disk space (and brainspace) by bringing the definition for turn_towards_heading out into a separate file. For example, suppose the Skipper figures out a half-dozen common subroutines related to navigating the Minnow that he seems to use in most or all of the programs he's writing for the task. He can put them in a separate file called navigation.pl, which consists only of the needed subroutines.

But now, how can you tell Perl to pull in that program snippet from another file? You could do it the hard way:

sub load_common_subroutines {
  open MORE_CODE, "navigation.pl" or die "navigation.pl: $!";
  undef $/; # enable slurp mode
  my $more_code = <MORE_CODE>;
  close MORE_CODE;
  eval $more_code;
  die $@ if $@;
}

The code from navigation.pl is read into the $more_code variable. You then use eval to process that text as Perl code. Any lexical variables in $more_code will remain local to the evaluated code. If there's a syntax error, the $@ variable is set and causes the subroutine to die with the appropriate error message.

Now instead of a few dozen lines of common subroutines to place in each file, you simply have one subroutine to insert in each file.

But that's not very nice, especially if you need to keep doing this kind of task repeatedly. Luckily, there's (at least) one Perl built-in to help you out.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Using do

Content preview·Buy reprint rights for this chapter

The Skipper has placed a few common navigation subroutines into navigation.pl. If the Skipper merely inserts:

do "navigation.pl";
die $@ if $@;

into his typical navigation program, it's almost the same as if the eval code were executed earlier.

That is, the do operator acts as if the code from navigation.pl were incorporated into the current program, although in its own scope block so that lexicals (my variables) and most directives (such as use strict) from the included file don't leak into the main program.

Now the Skipper can safely update and maintain only one copy of the common subroutines, without having to copy and recopy all the fixes and extensions into the many separate navigation programs he is creating and using. See Figure 2-1 for an illustration.

Figure 2-1: The navigation.pl file being used by the other navigation programs

Of course, this requires a bit of discipline because breaking the expected interface of a given subroutine will now break many programs instead of just one. Careful thought will need to be given as to how to design and write reusable components and modular design. We'll presume The Skipper has had some experience at that.

Another advantage to placing some of the code into a separate file is that other programmers can reuse the Skipper's routines and vice versa. For example, suppose the Skipper's sidekick (we'll call him "Gilligan") writes a routine to drop_anchor( ) and places it in the file drop_anchor.pl.

Then, the Skipper can use the code with:

do "drop_anchor.pl";
die $@ if $@;
...
drop_anchor(  ) if at_dock(  ) or in_port(  );

Thus, the code is brought into separate files to permit easy maintenance and interprogrammer cooperation.

While the code brought in from a .pl file can have direct executable statements, it's much more common to simply define subroutines that can be called by the code containing the do.

Going back to that drop_anchor.pl library for a second, imagine what would happen if the Skipper wrote a program that needed to "drop anchor" as well as navigate:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Using require

Content preview·Buy reprint rights for this chapter

Suppose navigate.pl itself also pulls in drop_anchor.pl for some common navigation task. You'll end up reading the file once directly, and then again while processing the navigation package. This will needlessly redefine drop_anchor( ). Worse than that, if warnings are enabled, you'll get a warning from Perl that you've redefined the subroutine, even though it's the same definition.

What you need is a mechanism that tracks what files have been brought in and bring them in only once. Perl has such an operation, called require. Change the previous code to simply:

require "drop_anchor.pl";
require "navigate.pl";

The require operator keeps track of the files it has read. Once a file has been processed successfully, any further require operations on that same file are simply ignored. This means that even if navigate.pl contains require "drop_anchor.pl", the drop_anchor.pl file is brought in exactly once, and you'll get no annoying error messages about duplicate subroutine definitions (see Figure 2-2). Most importantly, you'll also save time by not processing the file more than once .

Figure 2-2: Once the drop_anchor.pl file is brought in, another attempt to require the file is harmless

The require operator also has two additional features:

• Any syntax error in the required file causes the program to die, thus the many die $@ if $@ statements are unnecessary.
• The last expression evaluated in the file must return a true value.

Because of the second point, most files evaluated for require have a cryptic 1; as their last line of code. This ensures that the last evaluated expression is in fact true. Try to carry on this tradition as well.

Originally, the mandatory true value was intended as a way for an included file to signal to the invoker that the code was processed successfully and that no error condition existed. However, nearly everyone has adopted the die if ... strategy instead, deeming the "last expression evaluated is false" strategy a mere historic annoyance.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

require and @INC

Content preview·Buy reprint rights for this chapter

So far, the examples have glossed over the directory structure of where the main code and the included files (either with do or require) are located. That's because it "just works" for the simplest case, in which you have a program and its libraries in the same directory, and you run the program from that directory.

Things get a bit more complicated when the libraries aren't located in the current directory. In fact, Perl searches for libraries along a library search path (similar to what the shell does with the PATH environment variable). The current directory (represented in Unix by a single dot) is an element of the search path, so as long as your libraries are in your current working directory, everything is fine.

The search path is given in the special @INC array. By default, the array contains the current directory and a half-dozen directories built in to the perl binary during the compilation of perl itself. You can see what these directories are by typing perl -V at the command line and noting the last dozen lines of the output. Also at the command line, you can execute the following to get just the @INC directories:

perl -le 'print for @INC'

Except for . in that list, you probably won't be able to write to any of the other directories, unless you're the person responsible for maintaining Perl on your machine, in which case you should be able to write to all of them. The remaining directories are where Perl searches for system-wide libraries and modules, as you'll see later.

Although you may not be able to alter the content of the directories named by @INC, you can alter @INC itself before the require, to bring in libraries from one or more directories of your choosing. The @INC array is an ordinary array, so have the Skipper add a directory below his home directory to the mix:

unshift @INC, "/home/skipper/perl-lib";

Now, in addition to searching the standard directories and the current directory, Perl searches the Skipper's personal Perl library. In fact, Perl searches in that directory first, since it is the first one in

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

The Problem of Namespace Collisions

Content preview·Buy reprint rights for this chapter

Suppose that the Skipper has added all his cool and useful routines to navigation.pl and that Gilligan has incorporated the library into his own navigation package head_towards_island:

#!/usr/bin/perl

require 'navigation.pl';

sub turn_toward_port {
  turn_toward_heading(compute_heading_to_island(  ));
}

sub compute_heading_to_island {
  .. code here ..
}

.. more program here ..

Gilligan then has his program debugged (perhaps with the aid of a smart person whom we'll call "the Professor"), and everything works well.

However, now the Skipper decides to modify his navigation.pl library, adding a routine called turn_toward_port that makes a 45-degree turn toward the left (known as "port" in nautical jargon).

Gilligan's program will fail in a catastrophic way, as soon as he tries to head to port: he'll start steering the ship in circles! The problem is that the Perl compiler first compiles turn_toward_port from Gilligan's main program, then when the require is evaluated at runtime, the definition for turn_toward_port is redefined as the Skipper's definition. Sure, if Gilligan has warnings enabled, he'll notice something is wrong, but why should he have to count on that?

The problem is that Gilligan defined turn_toward_port as meaning "turn toward the port on the island," while the Skipper defined it as "turn toward the left." How do you resolve this?

One way is to require that the Skipper put an explicit prefix in front of every name defined in the library, say navigation_. Thus, Gilligan's program ends up looking like:

#!/usr/bin/perl

require 'navigation.pl';

sub turn_toward_port {
  navigation_turn_toward_heading(compute_heading_to_island(  ));
}

sub compute_heading_to_island {
  .. code here ..
}

.. more program here ..

Clearly, the navigation_turn_toward_heading comes from the navigation.pl file. This is great for Gilligan, but awkward for the Skipper, as his file now becomes:

sub navigation_turn_toward_heading {
  .. code here ..
}

sub navigation_turn_toward_port {
  .. code here ..
}

1;

Yes, every scalar, array, hash, filehandle, or subroutine now has to have a

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Packages as Namespace Separators

Content preview·Buy reprint rights for this chapter

If the name prefix of the last example didn't have to be spelled out on every use, things would work much better. Well, you can improve the situation by using a package:

package Navigation;

sub turn_towards_heading {
  .. code here ..
}

sub turn_towards_port {
  .. code here ..
}

1;

The package declaration at the beginning of this file tells Perl to insert Navigation:: in front of most names within the file: Thus, the code above practically says:

sub Navigation::turn_towards_heading {
  .. code here ..
}

sub Navigation::turn_towards_port {
  .. code here ..
}

1;

Now when Gilligan uses this file, he simply adds Navigation:: to the subroutines defined in the library, and leaves the Navigation:: prefix off for subroutines he defines on his own:

#!/usr/bin/perl

require 'navigation.pl';

sub turn_toward_port {
  Navigation::turn_toward_heading(compute_heading_to_island(  ));
}

sub compute_heading_to_island {
  .. code here ..
}

.. more program here ..

Package names are like variable names: they consist of alphanumerics and underscores, as long as you don't begin with a digit. Also, for reasons explained in the perlmodlib documentation, a package name should begin with a capital letter and not overlap an existing CPAN or core module name. Package names can also consist of multiple names separated by double colons, such as Minnow::Navigation and Minnow::Food::Storage.

Nearly every scalar, array, hash, subroutine, and filehandle name is actually prefixed by the current package, unless the name already contains one or more double-colon markers.

So, in navigation.pl, you can use variables such as:

package Navigation;
@homeport = (21.1, -157.525);

sub turn_toward_port {
  .. code ..
}

(Trivia note: 21.1 degrees north, 157.525 degrees west is the location of the real-life marina where the opening shot of a famous television series was filmed.)

You can refer to the @homeport variable in the main code as:

@destination = @Navigation::homeport;

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Scope of a Package Directive

Content preview·Buy reprint rights for this chapter

All files start as if you had said package main;. Any package directive remains in effect until the next package directive, unless that package directive is inside a curly-braced scope. In that case, the prior package is remembered and restored when the scope ends. Here's an example:

package Navigation;

{  # start scope block
  package main;  # now in package main

  sub turn_towards_heading {  # main::turn_towards_heading
    .. code here ..
  }

}  # end scope block

# back to package Navigation

sub turn_towards_port { # Navigation::turn_towards_port
  .. code here ..
}

The current package is lexically scoped, similar to the scope of my variables, narrowed to the innermost-enclosing brace pair or file in which the package is introduced.

Most libraries have only one package declaration at the top of the file. Most programs leave the package at the default main package. However it's nice to know that you can temporarily have a different current package.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Packages and Lexicals

Content preview·Buy reprint rights for this chapter

A lexical variable (a variable introduced with my) isn't prefixed by the current package because package variables are always global: you can always reference a package variable if you know its full name. A lexical variable is usually temporary and accessible for only a portion of the program. If a lexical variable is declared, then using that name without a package prefix results in accessing the lexical variable. However, a package prefix ensures that you are accessing a package variable and never a lexical variable.

For example, suppose a subroutine within navigation.pl declares a lexical @homeport variable. Any mention of @homeport will then be the newly introduced lexical variable, but a fully qualified mention of @Navigation::homeport accesses the package variable instead.

package Navigation;
@homeport = (21.1, -157.525);

sub get_me_home {
  my @homeport;

  .. @homeport .. # refers to the lexical variable
  .. @Navigation::homeport .. # refers to the package variable

}

.. @homeport .. # refers to the package variable

Obviously, this can lead to confusing code, so you shouldn't introduce such a duplication needlessly. The results are completely predictable, though.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Exercises

Content preview·Buy reprint rights for this chapter

The answers for all exercises can be found in Section A.1.

The Oogaboogoo natives on the island have unusual names for the days and months. Here is some simple but not very well-written code from Gilligan. Fix it up, add a conversion function for the month names, and make the whole thing into a library. For extra credit, add suitable error checking and consider what should be in the documentation.

@day = qw(ark dip wap sen pop sep kir);
sub number_to_day_name { my $num = shift @_; $day[$num]; }
@month = qw(diz pod bod rod sip wax lin sen kun fiz nap dep);

Make a program that uses your library and the following code to print out a message, such as Today is dip, sen 11, 2008, meaning that today is a Monday in August. (Hint: The year and month numbers returned by localtime may not be what you'd expect, so you need to check the documentation.)

my($sec, $min, $hour, $mday, $mon, $year, $wday) = localtime;

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 3: Introduction to References

Content preview·Buy reprint rights for this chapter

A Perl scalar variable holds a single value. An array holds an ordered list of one or more scalars. A hash holds a collection of scalars as values, keyed by other scalars.

Although a scalar can be an arbitrary string, which allows complex data to be encoded into an array or hash, none of the three data types are well-suited to complex data interrelationships. This is a job for the reference. Let's look at the importance of references by starting with an example.

Before the Minnow can leave on an excursion (e.g., a three-hour tour), every passenger and crew member should be checked to ensure they have all the required trip items in their possession. Let's say that for maritime safety, every person on board the Minnow needs to have a life preserver, some sunscreen, a water bottle, and a rain jacket. You can write a bit of code to check for the Skipper's supplies:

my @required = qw(preserver sunscreen water_bottle jacket);
my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
for my $item (@required) {
  unless (grep $item eq $_, @skipper) { # not found in list?
    print "skipper is missing $item.\n";
  }
}

The grep in a scalar context returns the number of times the expression $item eq $_ returns true, which is 1 if the item is in the list and 0 if not. If the value is 0, it's false, and you print the message.

Of course, if you want to check on Gilligan and the Professor, you might write the following code:

my @gilligan = qw(red_shirt hat lucky_socks water_bottle);
for my $item (@required) {
  unless (grep $item eq $_, @gilligan) { # not found in list?
    print "gilligan is missing $item.\n";
  }
}

my @professor = qw(sunscreen water_bottle slide_rule batteries radio);
for my $item (@required) {
  unless (grep $item eq $_, @professor) { # not found in list?
    print "professor is missing $item.\n";
  }
}

You may start to notice a lot of repeated code here and decide that it would be served best in a subroutine:

sub check_required_items {
  my $who = shift;
  my @required = qw(preserver sunscreen water_bottle jacket);
  for my $item (@required) {
    unless (grep $item eq $_, @_) { # not found in list?
      print "$who is missing $item.\n";
    }
  }
}

my @gilligan = qw(red_shirt hat lucky_socks water_bottle);
check_required_items("gilligan", @gilligan);

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Performing the Same Task on Many Arrays

Content preview·Buy reprint rights for this chapter

Before the Minnow can leave on an excursion (e.g., a three-hour tour), every passenger and crew member should be checked to ensure they have all the required trip items in their possession. Let's say that for maritime safety, every person on board the Minnow needs to have a life preserver, some sunscreen, a water bottle, and a rain jacket. You can write a bit of code to check for the Skipper's supplies:

my @required = qw(preserver sunscreen water_bottle jacket);
my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
for my $item (@required) {
  unless (grep $item eq $_, @skipper) { # not found in list?
    print "skipper is missing $item.\n";
  }
}

The grep in a scalar context returns the number of times the expression $item eq $_ returns true, which is 1 if the item is in the list and 0 if not. If the value is 0, it's false, and you print the message.

Of course, if you want to check on Gilligan and the Professor, you might write the following code:

my @gilligan = qw(red_shirt hat lucky_socks water_bottle);
for my $item (@required) {
  unless (grep $item eq $_, @gilligan) { # not found in list?
    print "gilligan is missing $item.\n";
  }
}

my @professor = qw(sunscreen water_bottle slide_rule batteries radio);
for my $item (@required) {
  unless (grep $item eq $_, @professor) { # not found in list?
    print "professor is missing $item.\n";
  }
}

You may start to notice a lot of repeated code here and decide that it would be served best in a subroutine:

sub check_required_items {
  my $who = shift;
  my @required = qw(preserver sunscreen water_bottle jacket);
  for my $item (@required) {
    unless (grep $item eq $_, @_) { # not found in list?
      print "$who is missing $item.\n";
    }
  }
}

my @gilligan = qw(red_shirt hat lucky_socks water_bottle);
check_required_items("gilligan", @gilligan);

The subroutine is given five items in its @_ array initially: the name gilligan and the four items belonging to Gilligan. After the shift, @_ will have only the items. Thus, the grep checks each required item against the list.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Taking a Reference to an Array

Content preview·Buy reprint rights for this chapter

Among its many other meanings, the backslash (\) character is also the "take a reference to" operator. When you use it in front of an array name, e.g., \@skipper, the result is a reference to that array. A reference to the array is like a pointer: it points at the array, but is not the array itself.

A reference fits wherever a scalar fits. It can go into an element of an array or a hash, or into a plain scalar variable, like this:

my $reference_to_skipper = \@skipper;

The reference can be copied:

my $second_reference_to_skipper = $reference_to_skipper;

or even:

my $third_reference_skipper = \@skipper;

All three references are completely interchangeable. You can even say they're identical:

if ($reference_to_skipper == $second_reference_to_skipper) {
  print "They are identical references.\n";
}

This equality compares the numeric forms of the two references. The numeric form of the reference is the unique memory address of the @skipper internal data structure, unchanging during the life of the variable. If you look at the string form instead, with eq or print, you get a debugging string:

ARRAY(0x1a2b3c)

which again is unique for this array because it includes the hexadecimal (base 16) representation of the array's unique memory address. The debugging string also notes that this is an array reference. Of course, if you ever see something like this in your output, it almost certainly means there's a bug; users of your program have little interest in hex dumps of storage addresses!

Because a reference can be copied, and passing an argument to a subroutine is really just copying, you can use this code to pass a reference to the array into the subroutine:

my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
check_required_items("The Skipper", \@skipper);

sub check_required_items {
  my $who = shift;
  my $items = shift;
  my @required = qw(preserver sunscreen water_bottle jacket);
  ...
}

Now $items

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Dereferencing the Array Reference

Content preview·Buy reprint rights for this chapter

If you look at @skipper, you'll see that it consists of two parts: the @ symbol and the name of the array. Similarly, the syntax $skipper[1] consists of the name of the array in the middle and some syntax around the outside to get at the second element of the array (index value 1 is the second element because you start counting index values at 0).

Here's the trick: any reference to an array can be placed in curly braces and written in place of the name of an array, ending up with a method to access the original array. That is, wherever you write skipper to name the array, you use the reference inside curly braces: { $items }. For example, both of these lines refer to the entire array:

@  skipper
@{ $items }

whereas both of these refer to the second item of the array:

$  skipper [1]
${ $items }[1]

By using the reference form, you've decoupled the code and the method of array access from the actual array. Let's see how that changes the rest of this subroutine:

sub check_required_items {
  my $who = shift;
  my $items = shift;
  my @required = qw(preserver sunscreen water_bottle jacket);
  for my $item (@required) {
    unless (grep $item eq $_, @{$items}) { # not found in list?
      print "$who is missing $item.\n";
    }
  }
}

All you did was replace @_ (the copy of the provisions list) with @{$items}, a dereferencing of the reference to the original provisions array. Now you can call the subroutine a few times as before:

my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
check_required_items("The Skipper", \@skipper);
my @professor = qw(sunscreen water_bottle slide_rule batteries radio);
check_required_items("Professor", \@professor);
my @gilligan = qw(red_shirt hat lucky_socks water_bottle);
check_required_items("Gilligan", \@gilligan);

In each case, $items points to a different array, so the same code applies to different arrays each time it is invoked. This is one of the most important uses of references: decoupling the code from the data structure on which it operates so the code can be reused more readily.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Dropping Those Braces

Content preview·Buy reprint rights for this chapter

Most of the time, the dereferenced array reference is contained in a simple scalar variable, such as @{$items} or ${$items}[1]. In those cases, the curly braces can be dropped, unambiguously, forming @$items or $$items[1].

However, the braces cannot be dropped if the value within the braces is not a simple scalar variable. For example, for @{$_[1]} from that last subroutine rewrite, you can't remove the braces.

This rule also means that it's easy to see where the "missing" braces need to go. When you see $$items[1], a pretty noisy piece of syntax, you can tell that the curly braces must belong around the simple scalar variable, $items. Therefore, $items must be a reference to an array.

Thus, an easier-on-the-eyes version of that subroutine might be:

sub check_required_items {
  my $who = shift;
  my $items = shift;
  my @required = qw(preserver sunscreen water_bottle jacket);
  for my $item (@required) {
    unless (grep $item eq $_, @$items) { # not found in list?
      print "$who is missing $item.\n";
    }
  }
}

The only difference here is that the braces were removed for @$items.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Modifying the Array

Content preview·Buy reprint rights for this chapter

You've seen how to solve the excessive copying problem with an array reference. Now let's look at modifying the original array.

For every missing provision, push that provision onto an array, forcing the passenger to consider the item:

sub check_required_items {
  my $who = shift;
  my $items = shift;
  my @required = qw(preserver sunscreen water_bottle jacket);
  my @missing = (  );

  for my $item (@required) {
    unless (grep $item eq $_, @$items) { # not found in list?
      print "$who is missing $item.\n";
      push @missing, $item;
    }
  }

  if (@missing) {
    print "Adding @missing to @$items for $who.\n";
    push @$items, @missing;
  }
}

Note the addition of the @missing array. If you find any items missing during the scan, push them into @missing. If there's anything there at the end of the scan, add it to the original provision list.

The key is in the last line of that subroutine. You're dereferencing the $items array reference, accessing the original array, and adding the elements from @missing. Without passing by reference, you'd modify only a local copy of the data, which has no effect on the original array.

Also, @$items (and its more generic form @{$items}) works within a double-quoted string. Do not include any whitespace between the @ and the immediately following character, although you can include nearly arbitrary whitespace within the curly braces as if it were normal Perl code.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Nested Data Structures

Content preview·Buy reprint rights for this chapter

In this example, the array @_ contains two elements, one of which is also an array. What if you take a reference to an array that also contains a reference to an array? You end up with a complex data structure, which can be quite useful.

For example, iterate over the data for the Skipper, Gilligan, and the Professor by first building a larger data structure holding the entire list of provision lists:

my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
my @skipper_with_name = ("Skipper", \@skipper);
my @professor = qw(sunscreen water_bottle slide_rule batteries radio);
my @professor_with_name = ("Professor", \@professor);
my @gilligan = qw(red_shirt hat lucky_socks water_bottle);
my @gilligan_with_name = ("Gilligan", \@gilligan);

At this point, @skipper_with_name has two elements, the second of which is an array reference, similar to what was passed to the subroutine. Now group them all:

my @all_with_names = (
  \@skipper_with_name,
  \@professor_with_name,
  \@gilligan_with_name,
);

Note that you have just three elements, each of which is a reference to an array, each of which has two elements: the name and its corresponding initial provisions. A picture of that is in Figure 3-1.

Figure 3-1: The array @all_with_names holds a multilevel data structure containing strings and references to arrays

Therefore, $all_with_names[2] will be the array reference for the Gilligan's data. If you dereference it as @{$all_with_names[2]}, you get a two-element array, "Gilligan" and another array reference.

How would you access that array reference? Using your rules again, it's ${$all_with_names[2]}[1]. In other words, taking $all_with_names[2], you dereference it in an expression that would be something like $DUMMY[1] as an ordinary array, so you'll place {$all_with_names[2]} in place of DUMMY.

How do you call the existing check_required_items( ) with this data structure? The following code is easy enough.

for my $person (@all_with_names) {
  my $who = $$person[0];
  my $provisions_reference = $$person[1];
  check_required_items($who, $provisions_reference);
}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Simplifying Nested Element References with Arrows

Content preview·Buy reprint rights for this chapter

Look at the curly-brace dereferencing again. As in the earlier example, the array reference for Gilligan's provision list is ${$all_with_names[2]}[1]. Now, what if you want to know Gilligan's first provision? You need to dereference this item one more level, so it's Yet Another Layer of Braces: ${${$all_with_names[2]}[1]}[0]. That's a really noisy piece of syntax. Can you shorten that? Yes!

Everywhere you write ${DUMMY}[$y], you can write DUMMY->[$y] instead. In other words, you can dereference an array reference, picking out a particular element of that array by simply following the expression defining the array reference with an arrow and a square-bracketed subscript.

For this example, this means you can pick out the array reference for Gilligan with a simple $all_with_names[2]->[1], and Gilligan's first provision with $all_with_names[2]->[1]->[0]. Wow, that's definitely easier on the eyes.

If that wasn't already simple enough, there's one more rule: if the arrow ends up between "subscripty kinds of things," like square brackets, you can also drop the arrow. $all_with_names[2]->[1]->[0] becomes $all_with_names[2][1][0]. Now it's looking even easier on the eye.

The arrow has to be between subscripty things. Why wouldn't it be between? Well, imagine a reference to the array @all_with_names:

my $root = \@all_with_names;

Now how do you get to Gilligan's first item?

$root -> [2] -> [1] -> [0]

More simply, using the "drop arrow" rule, you can use:

$root -> [2][1][0]

You cannot drop the first arrow, however, because that would mean an array @root's third element, an entirely unrelated data structure. Let's compare this to the full curly-brace form again:

${${${$root}[2]}[1]}[0]

It looks much better with the arrow. Note, however, that no shortcut gets the entire array from an array reference. If you want all of Gilligan's provisions, you say:

@{$root->[2][1]}

Reading this from the inside out, you can think of it like this:

1. Take $root.
2. Dereference it as an array reference, taking the third element of that array (index number 2).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

References to Hashes

Content preview·Buy reprint rights for this chapter

Just as you can take a reference to an array, you can also take a reference to a hash. Once again, you use the backslash as the "take a reference to" operator:

my %gilligan_info = (
  name => 'Gilligan',
  hat => 'White',
  shirt => 'Red',
  position => 'First Mate',
);
my $hash_ref = \%gilligan_info;

You can dereference a hash reference to get back to the original data. The strategy is similar to dereferencing an array reference. Write the hash syntax as you would have without references, and then replace the name of the hash with a pair of curly braces surrounding the thing holding the reference. For example, to pick a particular value for a given key, use:

my $name = $ gilligan_info { 'name' };
my $name = $ { $hash_ref } { 'name' };

In this case, the curly braces have two different meanings. The first pair denotes the expression returning a reference, while the second pair delimits the expression for the hash key.

To perform an operation on the entire hash, you proceed similarly:

my @keys = keys % gilligan_info;
my @keys = keys % { $hash_ref };

As with array references, you can use shortcuts to replace the complex curly-braced forms under some circumstances. For example, if the only thing inside the curly braces is a simple scalar variable (as shown in these examples so far), you can drop the curly braces:

my $name = $$hash_ref{'name'};
my @keys = keys %$hash_ref;

Like an array reference, when referring to a specific hash element, you can use an arrow form:

my $name = $hash_ref->{'name'};

Because a hash reference fits wherever a scalar fits, you can create an array of hash references:

my %gilligan_info = (
  name => 'Gilligan',
  hat => 'White',
  shirt => 'Red',
  position => 'First Mate',
);
my %skipper_info = (
  name => 'Skipper',
  hat => 'Black',
  shirt => 'Blue',
  position => 'Captain',
);
my @crew = (\%gilligan_info, \%skipper_info);

Thus, $crew[0] is a hash reference to the information about Gilligan. You can get to Gilligan's name via any one of:

${ $crew[0] } { 'name' }
my $ref = $crew[0]; $$ref{'name'}
$crew[0]->{'name'}
$crew[0]{'name'}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Exercises

Content preview·Buy reprint rights for this chapter

The answers for all exercises can be found in Section A.2.

How many different things do these expressions refer to?

$ginger->[2][1]
${$ginger[2]}[1]
$ginger->[2]->[1]
${$ginger->[2]}[1]

Using the final version of check_required_items, write a subroutine check_items_for_all that takes a hash reference as its only parameter, pointing at a hash whose keys are the people aboard the Minnow, and whose corresponding values are array references of the things they intend to bring on board.

For example, the hash reference might be constructed like so:

my @gilligan = ... gilligan items ...;
my @skipper = ... skipper items ...;
my @professor = ... professor items ...;
my %all = (
  "Gilligan" => \@gilligan,
  "Skipper" => \@skipper,
  "Professor" => \@professor,
);
check_items_for_all(\%all);

The newly constructed subroutine should call check_required_items for each person in the hash, updating their provisions list to include the required items.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 4: References and Scoping

Content preview·Buy reprint rights for this chapter

References can be copied and passed around like any other scalar. At any given time, Perl knows the number of references to a particular data item. Perl can also create references to anonymous data structures (structures that do not have explicit names) and create references automatically as needed to fulfill certain kinds of operations. Let's look at copying references and how it affects scoping and memory usage.

Chapter 3 explored how to take a reference to an array @skipper and place it into a new scalar variable:

my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
my $reference_to_skipper = \@skipper;

You can then copy the reference or take additional references, and they'd all refer to the same thing and be interchangeable:

my $second_reference_to_skipper = $reference_to_skipper;
my $third_reference_to_skipper = \@skipper;

At this point, you have four different ways to access the data contained in @skipper:

@skipper
@$reference_to_skipper
@$second_reference_to_skipper
@$third_reference_to_skipper

Perl tracks how many ways the data can be accessed through a mechanism called reference counting. The original name counts as one, and each additional reference that was taken (including copies of references) also counts as one. The total number of references to the array of provisions is now four.

You can add and remove references as you wish, and as long as the reference count doesn't hit zero, the array is maintained in memory and is still accessible via any of the other access paths. For example, you might have a temporary reference:

check_provisions_list(\@skipper)

When this subroutine begins executing, a fifth reference to the data is created and copied into @_ for the subroutine. The subroutine is free to create additional copies of that reference, which Perl notes as needed. Typically, when the subroutine returns, all such references are discarded automatically, and you're back to four references again.

You can kill off each reference by using the variable for something other than a reference to the value of

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

More than One Reference to Data

Content preview·Buy reprint rights for this chapter

Chapter 3 explored how to take a reference to an array @skipper and place it into a new scalar variable:

my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
my $reference_to_skipper = \@skipper;

You can then copy the reference or take additional references, and they'd all refer to the same thing and be interchangeable:

my $second_reference_to_skipper = $reference_to_skipper;
my $third_reference_to_skipper = \@skipper;

At this point, you have four different ways to access the data contained in @skipper:

@skipper
@$reference_to_skipper
@$second_reference_to_skipper
@$third_reference_to_skipper

Perl tracks how many ways the data can be accessed through a mechanism called reference counting. The original name counts as one, and each additional reference that was taken (including copies of references) also counts as one. The total number of references to the array of provisions is now four.

You can add and remove references as you wish, and as long as the reference count doesn't hit zero, the array is maintained in memory and is still accessible via any of the other access paths. For example, you might have a temporary reference:

check_provisions_list(\@skipper)

When this subroutine begins executing, a fifth reference to the data is created and copied into @_ for the subroutine. The subroutine is free to create additional copies of that reference, which Perl notes as needed. Typically, when the subroutine returns, all such references are discarded automatically, and you're back to four references again.

You can kill off each reference by using the variable for something other than a reference to the value of @skipper. For example, you can assign undef to the variable:

$reference_to_skipper = undef;

Or, maybe just let the variable go out of scope:

my @skipper = ...;
{
  ...
  my $ref = \@skipper;
  ...
  ...
} # $ref goes out of scope at this point

In particular, a reference held in a subroutine's private (lexical) variable goes away at the end of the subroutine.

Whether the value is changed or the variable itself goes away, Perl notes it as an appropriate reduction in the number of references to the data.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What if That Was the Name?

Content preview·Buy reprint rights for this chapter

Typically, all references to a variable are removed before the variable itself. But what if one of the references outlives the variable name? For example, consider this code:

my $ref;
{
  my @skipper = qw(blue_shirt hat jacket preserver sunscreen);
  $ref = \@skipper;
  print "$ref->[2]\n"; # prints jacket\n
}
print "$ref->[2]\n"; # still prints jacket\n

Immediately after the @skipper array is declared, you have one reference to the five-element list. After $ref is initialized, you'll have two, down to the end of the block. When the block ends, the @skipper name disappears. However, this was only one of the two ways to access the data! Thus, the five-element list is not removed from memory, and $ref is still pointing to that data.

At this point, the five-element list is contained within an anonymous array, which is a fancy term for an array without a name.

Until the value of $ref is changed, or $ref itself disappears, you can still continue to use all the dereferencing strategies you used prior to when the name of the array disappeared. In fact, it's still a fully functional array that you can shrink or grow just as you do any other Perl array:

push @$ref, "sextant"; # add a new provision
print "$ref->[-1]\n"; # prints sextant\n

You can even increase the reference count at this point:

my $copy_of_ref = $ref;

or equivalently:

my $copy_of_ref = \@$ref;

The data remains alive until the last reference is destroyed:

$ref = undef; # not yet...
$copy_of_ref = undef; # poof!

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Reference Counting and Nested Data Structures

Content preview·