Practical Development Environments

Chapter 1. Introduction

Projects have Environments in which People produce Products.

“Pithy, but painful,” was a friend’s wry reply when I asked his opinion of the above epigraph as a summary for this book. At a very high level, it really does describe the creation of software. The key word is environments, which in this book refers to all the tools used to produce software and how they are used, but not the details of how the software is written, or even what language the software is written in. For this book, an environment includes the software configuration management (SCM) tools, build tools, testing and bug tracking tools, and the release and maintenance processes. All these areas are covered in subsequent chapters of this book.

This chapter begins with an overview of how a software product is typically created. If you are involved in this process from day to day, you will probably be familiar with most of the steps. The amount of work in each step is different for different projects, groups of people, or companies, but all the steps appear in some guise. This chapter also briefly describes the difference between open and closed software.

The chapter then continues with some uncomfortable software development mistakes that most developers have stumbled across; avoiding these costly mistakes justifies reading the rest of this book.

Developing Software Products

Figure 1-1 shows the different activities that are generally involved in creating a software product, though it’s only a loose description of what really happens when developing software. The activities probably don’t occur in the same order in your experience, and each one is often repeated many times as a product is developed and maintained. You can probably also add some other arrows from experience with different projects that you’ve worked on. Some projects may not even have had all their activities connected, which is one of the things that a good development environment can help you with.

Figure 1-1. Typical activities involved in creating software products

Large or small, formal or more relaxed, open or closed (see Open and Closed Software Development, later in this chapter), all software projects have activities that resemble those shown in Figure 1-1. A short description of each activity follows:

Product marketing

This goes by many names, but whatever it is called, it always involves creating the ideas for the product. A subtle blend of technical knowledge about what has been done and what might be possible is combined with some imaginative flair for what might actually be wanted by potential customers. This is different from general marketing, which is described below.

Gathering requirements

The process of collecting information to clearly communicate to everyone in the project exactly what the product is supposed to do. Some different ways that this information can be described include documents, prototypes, and examples of how the product would be used.

Writing functional specifications

Functional specifications describe how the product can be created to fulfill the requirements.

Implementation

The gritty edge, where the product is actually created. A surprisingly small amount of the total time to develop a product is usually spent here.

Testing

This is where the implementation is compared with the requirements. A surprisingly large amount of the total time to develop a product is usually spent here, and yet this is commonly the first thing removed from busy schedules.

Documentation

Documentation describes how to use the product. Internal documentation may also describe how to maintain the product for future developers. Often written by a group called something like Technical Publications.

Release

The process of shipping the product to customers and making sure that the product will work properly outside the environment in which it was developed.

Marketing

Creating and controlling the market for the product. It also involves changing the product to suit the market, which connects back to the product marketing stage.

Sales

Persuading customers to use the product, often with the expectation that they will pay money for it.

Support

Helps the customers use the product when the documentation doesn’t answer their questions. Support also provides a contact point for any licenses needed to use the product, and may cover training as well.

Maintenance

If the product is commercial or is widely used, then as the world around it changes, the product will need work to keep it functioning properly and hence producing more revenue and happy customers.

End of support

Eventually no one wants to buy or use the product, not even the developers. Customers need to be told when product support will end, and the whole development environment and source files need to be carefully archived.

Some projects have particularly small or nonexistent activities, or the different activities may be connected very differently. For instance, the requirements document in a small startup company can often be the result of a late-evening “wouldn’t it be cool if...” session. A large project with hundreds of developers may have rigid requirements, with regular quarterly meetings to discuss changes to the design documents. Some activities not mentioned above include those performed by management, IT staff, and the legal department—all necessary overhead.

Open and Closed Software Development

Traditionally, software development is closed. This means that whether or not users are charged for the actual product, the source files that are used to create the product are confidential. Sometimes you can purchase the source, but that’s extra.

Open software development refers to a number of different ideas about software development. The best known one is that the source files for the product are openly and freely available, so customers can change the source files and then rebuild the product themselves, at least in theory. The source files may also be free, as in costing nothing. Many open source projects also allow anyone to redistribute the source files, subject to the requirement that if you make changes, you must make those changes publicly available. If you continue to distribute the changes you make to a product, then you may have produced a fork of the project, and the product will have become two different products. Forks are an accepted part of open software development.

Examples of some well-known open source software products are shown in Table 1-1.

Table 1-1. Major open software projects

Software	URL	Description
Apache	http://www.apache.org	Most of the world’s web sites run on the Apache web server.
GCC	http://gcc.gnu.org	The most widely used compiler for C and other languages.
GNU/Linux[1]	http://www.linux.org	The best-known open source Unix-based operating system.
Mozilla	http://www.mozilla.org	The Firefox web browser and many other Mozilla projects that grew out of the original Netscape browser.
Perl	http://www.perl.org	Powerful scripting languages, also known as dynamic languages.
Python	http://www.python.org
Tcl	http://tcl.sourceforge.net
[1] The term GNU/Linux is generally used instead of Linux throughout this book, since almost every Linux distribution is shipped with GNU tools.

The idea of open source development was pioneered by the Free Software Foundation, or FSF (http://www.fsf.org), founded by Richard Stallman in 1984; it has produced hundreds of open source products under the banner of the GNU Project (http://www.gnu.org). The FSF prefers to use the term free rather than open for this kind of software.

The Open Source Initiative, or OSI (http://www.opensource.org), was founded in 1998 and describes itself as a marketing program for free software. The OSI tends to distance itself from some of the philosophical positions of the FSF. As might be inferred from the name, the OSI uses the term open for this kind of software. Eric Raymond is one of the best-known people associated with the OSI.

Tip

You may also come across the compromise acronym F/OSS for “free/open source software,” or the terms gratis for “free as in beer” (no cost) and libre for “free as in freedom.”

There are a confusing number of different licenses for open source software. The best-known one is the GNU General Public License (GPL) from the FSF. The most common confusion about the GPL seems to be whether using source files that were distributed under the GPL means that the rest of a product’s source files must be made publicly available. It all seems to depend on how the GPL-licensed source files are used by the proprietary part of the product. I am not a lawyer, but there’s a useful book on this subject by someone who is: Understanding Open Source and Free Software Licensing, by Andrew M. St. Laurent (O’Reilly).

Regardless of whether a project is open or closed, each of the activities listed earlier in Developing Software Products is present. Customers for closed software are people and companies who have paid money to use the product. Customers for open software are people and companies who use the product, and they may well pay for support. Both kinds of software have to track the different legal licenses used for different parts of the product. Development environments for closed and open software projects often differ only in how much money will be spent on tools.

Although this book contains many references to open source projects and open source tools, creating a good development environment is about choosing the most appropriate tools. Both closed and open source tools are discussed in this book.

Dirty Secrets of Software Projects

Software projects can fail for many reasons, and one of them is a bad development environment. Here is an informal list of some frustrations and embarrassing mistakes commonly made in producing software. None of these have anything to do with failures caused by things like missing specifications, specifications changing over time (the dreaded “feature creep”), poor estimates of how long a project will take, or even whether the problem being solved is a really difficult one. Each of these problems is due to the tools or processes of the environment in which the software was produced:

• We can’t rebuild the same product that we shipped to the customer, so we’ll ship her the latest version, just to be safe.

• Building from scratch (also known as a clean build) each time is often the only way to get the product to build reliably. This takes so long that the developer goes off and does something else. Consequently, every developer thinks that his build tool is too slow.

• Testers, technical writers, and managers can’t build the latest version of the product. Even if it has already been built for them by someone else, it’s unclear where to find the latest version.

• Fixes for known bugs are not released, because it’s too hard to properly test the required changes and they may break other parts of the product.

• Everyone finds the bug tracking tool awkward to use.

• The environment used by developers and the one in which the product is used by customers are different in some important ways, so the developers never actually see problems the same way that a customer does.

• Getting a product ready to release takes so long that any late-arriving changes don’t get tested or documented.

• The wrong set of bits gets shipped to the customer. This one should make anyone involved in developing software wince, but it does happen.

• Nobody knows when to remove a feature from a product, because no one knows whether the feature is actually being used.

• Communication between groups happens by reading printouts left at the printer nearest to the coffee machine.

Sadly, the list could go on and on. Its contents are obviously subjective, but each entry has happened in real projects. The good news is that none of these Dirty Secrets are inevitable and they can all be eliminated with careful attention to the development environment. The promise of this book is that you can stop these kinds of embarrassments from happening to your project.

What Does “Practical” Mean?

What makes something practical? The title of this book strongly suggests that its contents are about practical solutions to common problems in development environments. Here is what practical means for the tools used in development environments:

• Available (whether open or closed source)

• Appropriately priced (for open software, this often means no cost)

• Usable—installation and configuration to match the local development process is possible, the tool doesn’t crash regularly, there aren’t too many bugs, and documentation is adequate

• Can eventually produce software for the platforms used by the customer

On the other hand, impractical tools for development environments are those that are:

• Dead—no longer sold or supported

• Too expensive to even consider spending any time evaluating

• Tortuous to get working or configured as needed, or impossible to maintain and upgrade

Abstract frameworks, conceptual models, and design patterns are all useful for categorizing solutions to problems encountered in development environments, but the aim of this book is to provide ideas that are more practical. The basic areas of projects that are summarized in the next chapter are ideas that are applicable to all software projects, whatever the product’s purpose is and whoever the customers are.

A Personal Tools Quiz

To make the ideas in this book seem more personal, try answering the following questions (honestly):

• Name the biggest mistake that you, personally, made in each of your last three projects. Program design doesn’t count; tools and processes do.

• Name the biggest mistake you think someone else made in your last three projects.

• Imagine that you could totally replace just one software tool that you use daily. Which one would it be, and why?

• Which tool is so good that you would have to be seriously bribed to be persuaded to stop using it?

• Which part of producing software just seems to take much longer than it should? Do you feel that you really understand what goes on in that part?

Your answers should give you some good ideas about which parts of this book will particularly interest you. It was questions like these, and a lack of written guidance on addressing the issues raised by their answers, that made me want to write this book in the first place.