Web Development with Node and Express

Chapter 4. Tidying Up

In the last two chapters, we were just experimenting: dipping our toes into the waters, so to speak. Before we proceed to more complex functionality, we’re going to do some housekeeping and build some good habits into our work.

In this chapter, we’ll start our Meadowlark Travel project in earnest. Before we start building the website itself, though, we’re going to make sure we have the tools we need to produce a high-quality product.

Tip

The running example in this book is not necessarily one you have to follow. If you’re anxious to build your own website, you could follow the framework of the running example, but modify it accordingly so that by the time you finish this book, you could have a finished website!

Best Practices

The phrase “best practices” is one you hear thrown around a lot these days, and it means that you should “do things right” and not cut corners (we’ll talk about what this means specifically in a moment). No doubt you’ve heard the engineering adage that your options are “fast,” “cheap,” and “good,” and you can pick any two. The thing that’s always bothered me about this model is that it doesn’t take into account the accrual value of doing things correctly. The first time you do something correctly, it may take five times as long to do it as it would have to do it quick and dirty. The second time, though, it’s only going to take three times as long. By the time you’ve done it correctly a dozen times, you’ll be doing it almost as fast as the quick and dirty way.

I had a fencing coach who would always remind us that practice doesn’t make perfect: practice makes permanent. That is, if you do something over and over again, eventually it will become automatic, rote. That is true, but it says nothing about the quality of the thing you are practicing. If you practice bad habits, then bad habits become rote. Instead, you should follow the rule that perfect practice makes perfect. In that spirit, I encourage you to follow the rest of the examples in this book as if you were making a real-live website, as if your reputation and remuneration were depending on the quality of the outcome. Use this book to not only learn new skills, but to practice building good habits.

The practices we will be focusing on are version control and QA. In this chapter, we’ll be discussing version control, and we’ll discuss QA in the next chapter.

Version Control

Hopefully I don’t have to convince you of the value of version control (if I did, that might take a whole book itself). Broadly speaking, version control offers these benefits:

Documentation

Being able to go back through the history of a project to see the decisions that were made and the order in which components were developed can be valuable documentation. Having a technical history of your project can be quite useful.

Attribution

If you work on a team, attribution can be hugely important. Whenever you find something in code that is opaque or questionable, knowing who made that change can save you many hours. It could be that the comments associated with the change are sufficient to answer your questions, and if not, you’ll know who to talk to.

Experimentation

A good version control system enables experimentation. You can go off on a tangent, trying something new, without fear of affecting the stability of your project. If the experiment is successful, you can fold it back into the project, and if it is not successful, you can abandon it.

Years ago, I made the switch to distributed version control systems (DVCS). I narrowed my choices down to Git and Mercurial, and went with Git, due to its ubiquity and flexibility. Both are excellent and free version control systems, and I recommend you use one of them. In this book, we will be using Git, but you are welcome to substitute Mercurial (or another version control system altogether).

If you are unfamiliar with Git, I recommend Jon Loeliger’s excellent Version Control with Git (O’Reilly). Also, Code School has a nice introductory course on Git.

How to Use Git with This Book

First, make sure you have Git. Type git --version. If it doesn’t respond with a version number, you’ll need to install Git. See the Git documentation for installation instructions.

There are two ways to follow along with the examples in this book. One is to type out the examples yourself, and follow along with the Git commands. The other is to clone the Git repository I am using for all of the examples and check out the associated tags for each example. Some people learn better by typing out examples, while some prefer to just see and run the changes without having to type it all in.

If You’re Following Along by Doing It Yourself

We’ve already got a very rough framework for our project: some views, a layout, a logo, a main application file, and a package.json file. Let’s go ahead and create a Git repository and add all those files.

First, we go to the project directory and create a Git repository there:

git init

Now before we add all the files, we’ll create a .gitignore file to help prevent us from accidentally adding things we don’t want to add. Create a text file called .gitignore in your project directory in which you can add any files or directories you want Git to ignore by default (one per line). It also supports wildcards. For example, if your editor creates backup files with a tilde at the end (like meadowlark.js~), you might put *~ in the .gitignore file. If you’re on a Mac, you’ll want to put .DS_Store in there. You’ll also want to put node_modules in there (for reasons that will be discussed soon). So for now, the file might look like this:

node_modules
*~
.DS_Store

Note

Entries in the .gitignore file also apply to subdirectories. So if you put *~ in the .gitignore in the project root, all such backup files will be ignored even if they are in subdirectories.

Now we can add all of our existing files. There are many ways to do this in Git. I generally favor git add -A, which is the most sweeping of all the variants. If you are new to Git, I recommend you either add files one by one (git add meadowlark.js, for example) if you only want to commit one or two files, or git add -A if you want to add all of your changes (including any files you might have deleted). Since we want to add all the work we’ve already done, we’ll use:

git add -A

Tip

Newcomers to Git are commonly confused by the git add command: it adds changes, not files. So if you’ve modified meadowlark.js, and then you type git add meadowlark.js, what you’re really doing is adding the changes you’ve made.

Git has a “staging area,” where changes go when you run git add. So the changes we’ve added haven’t actually been committed yet, but they’re ready to go. To commit the changes, use git commit:

git commit -m "Initial commit."

The -m "Initial commit." allows you to write a message associated with this commit. Git won’t even let you make a commit without a message, and for good reason. Always strive to make meaningful commit messages: they should briefly but concisely describe the work you’ve done.

If You’re Following Along by Using the Official Repository

To get the official repository for this book, run git clone:

git clone https://github.com/EthanRBrown/web-development-with-node-and-express

This repository has a directory for each chapter that contains code samples. For example, the source code for this chapter can be found in the ch04 directory. These directories represent the end of the chapter. That is, when you finish this chapter, you can see the finished code in the ch04 directory. For chapters where there’s a significant divergence, there may be additional directories, such as ch08-jquery-file-upload, which will be noted in the chapter text.

As this book is updated and improved, the repository will also be updated, and when it is, I will add a version tag so you can check out a version of the repository that corresponds to the version of the book you’re reading now. The current version of the repository is 1.5.1. If you have an older version of this book, and you wish to see the latest changes and improvements to the code samples, you can check out the most current version: just be warned that the repository samples will be different than what you read here.

Note

In the first version of this book, I took a different approach with the repository, with a linear history as if you were developing an increasingly sophisticated project. While this approach pleasantly mirrored the way a project in the real world might develop, it caused a lot of headache, both for me and my readers. As npm packages changed, the code samples would change, and short of rewriting the entire history of the repo, there was no good way to update the repository, or note the changes in the text. While the chapter-per-directory approach is more artificial, it allows the text to be synced more closely with the repository, and also enables easier community contribution.

Note

If at any point you want to experiment, keep in mind that the tag you have checked out puts you in what Git calls a “detached HEAD” state. While you are free to edit any files, it is unsafe to commit anything you do without creating a branch first. So if you do want to base an experimental branch off of a tag, simply create a new branch and check it out, which you can do with one command: git checkout -b experiment (where experiment is the name of your branch; you can use whatever you want). Then you can safely edit and commit on that branch as much as you want.

npm Packages

The npm packages that your project relies on reside in a directory called node_modules (it’s unfortunate that this is called node_modules and not npm_packages, as Node modules are a related but different concept). Feel free to explore that directory to satisfy your curiosity or to debug your program, but you should never modify any code in this directory. In addition to that being bad practice, all of your changes could easily be undone by npm. If you need to make a modification to a package your project depends on, the correct course of action would be to create your own fork of the project. If you do go this route, and you feel that your improvements would be useful to others, congratulations: you’re now involved in an open source project! You can submit your changes, and if they meet the project standards, they’ll be included in the official package. Contributing to existing packages and creating customized builds is beyond the scope of this book, but there is a vibrant community of developers out there to help you if you want to contribute to existing packages.

The purpose of the package.json file is twofold: to describe your project and to list dependencies. Go ahead and look at your package.json file now. You should see something like this (the exact version numbers will probably be different, as these packages get updated often):

{
  "dependencies": {
    "express": "^4.12.3",
    "express-handlebars": "^2.0.1"
  }
}

Right now, our package.json file contains only information about dependencies. The caret (^) in front of the package versions indicates that any version that starts with the specified version number—up to the next major version number—will work. For example, this package.json indicates that any version of Express that starts with 4.0.0 will work, so 4.0.1 and 4.9.9 would both work, but 3.4.7 would not, nor would 5.0.0. This is the default version specificity when you use npm install --save, and is generally a pretty safe bet. The consequence of this approach is that if you want to move up to a newer version, you will have to edit the file to specify the new version. Generally, that’s a good thing because it prevents changes in dependencies from breaking your project without your knowing about it. Version numbers in npm are parsed by a component called “semver” (for “semantic versioner”). If you want more information about versioning in npm, consult the semver documentation.

Since the package.json file lists all the dependencies, the node_modules directory is really a derived artifact. That is, if you were to delete it, all you would have to do to get the project working again would be to run npm install, which will recreate the directory and put all the necessary dependencies in it. It is for this reason that I recommend putting node_modules in your .gitignore file, and not including it in source control. However, some people feel that your repository should contain everything necessary to run the project, and prefer to keep node_modules in source control. I find that this is “noise” in the repository, and I prefer to omit it.

Whenever you use a Node module in your project, you should make sure it’s listed as a dependency in package.json. If you fail to do this, npm will be unable to construct the right dependencies, and when another developer checks out the project (or when you do on a different computer), the correct dependencies won’t be installed, which negates the value of a package manager.

Project Metadata

The other purpose of the package.json file is to store project metadata, such as the name of the project, authors, license information, and so on. If you use npm init to initially create your package.json file, it will populate the file with the necessary fields for you, and you can update them at any time. If you intend to make your project available on npm or GitHub, this metadata becomes critical. If you would like more information about the fields in package.json, see the package.json documentation. The other important piece of metadata is the README.md file. This file can be a handy place to describe the overall architecture of the website, as well as any critical information that someone new to the project might need. It is in a text-based wiki format called Markdown. Refer to the Markdown documentation for more information.

Node Modules

As mentioned earlier, Node modules and npm packages are related but different concepts. Node modules, as the name implies, offer a mechanism for modularization and encapsulation. npm packages provide a standardized scheme for storing, versioning, and referencing projects (which are not restricted to modules). For example, we import Express itself as a module in our main application file:

var express = require('express');

require is a Node function for importing a module. By default, Node looks for modules in the directory node_modules (it should be no surprise, then, that there’s an express directory inside of node_modules). However, Node also provides a mechanism for creating your own modules (you should never create your own modules in the node_modules directory). Let’s see how we can modularize the fortune cookie functionality we implemented in the previous chapter.

First let’s create a directory to store our modules. You can call it whatever you want, but lib (short for “library”) is a common choice. In that folder, create a file called fortune.js:

var fortuneCookies = [
        "Conquer your fears or they will conquer you.",
        "Rivers need springs.",
        "Do not fear what you don't know.",
        "You will have a pleasant surprise.",
        "Whenever possible, keep it simple.",
];

exports.getFortune = function() {
        var idx = Math.floor(Math.random() * fortuneCookies.length);
        return fortuneCookies[idx];
};

The important thing to note here is the use of the global variable exports. If you want something to be visible outside of the module, you have to add it to exports. In this example, the function getFortune will be available from outside this module, but our array fortuneCookies will be completely hidden. This is a good thing: encapsulation allows for less error-prone and fragile code.

Note

There are several ways to export functionality from a module. We will be covering different methods throughout the book and summarizing them in Chapter 22.

Now in meadowlark.js, we can remove the fortuneCookies array (though there would be no harm in leaving it: it can’t conflict in any way with the array with the same name defined in lib/fortune.js). It is traditional (but not required) to specify imports at the top of the file, so at the top of the meadowlark.js file, add the following line:

var fortune = require('./lib/fortune.js');

Note that we prefix our module name with ./. This signals to Node that it should not look for the module in the node_modules directory; if we omitted that prefix, this would fail.

Now in our route for the About page, we can utilize the getFortune method from our module:

app.get('/about', function(req, res) {
        res.render('about', { fortune: fortune.getFortune() } );
});

If you’re following along, let’s commit those changes:

git add -A
git commit -m "Moved 'fortune cookie' functionality into module."

Or if you’re using the official repository, you can see the changes in this tag:

git checkout ch04

You will find modules to be a very powerful and easy way to encapsulate functionality, which will improve the overall design and maintainability of your project, as well as make testing easier. Refer to the official Node module documentation for more information.