97 Things Every Programmer Should Know

Chapter 48. Large, Interconnected Data Belongs to a Database

Diomidis Spinellis

IF YOUR APPLICATION is going to handle a large, persistent, interconnected set of data elements, don’t hesitate to store it in a relational database. In the past, RDBMSs used to be expensive, scarce, complex, and unwieldy beasts. This is no longer the case. Nowadays, RDBMS systems are easy to find—it is likely that the system you’re using already has one or two installed. Some very capable RDBMSs, like MySQL and PostgreSQL, are available as open source software, so cost of purchase is no longer an issue. Even better, so-called embedded database systems can be linked as libraries directly into your application, requiring almost no setup or management—two notable open source ones are SQLite and HSQLDB. These systems are extremely efficient.

If your application’s data is larger than the system’s RAM, an indexed RDBMS table will perform orders of magnitude faster than your library’s map collection type, which will thrash virtual memory pages. Modern database offerings can easily grow with your needs. With proper care, you can scale up an embedded database to a larger database system when required. Later on, you can switch from a free, open source offering to a better-supported or more powerful proprietary system.

Once you get the hang of SQL, writing database-centric applications is a joy. After you’ve stored ...