Many practical systems need to store and retrieve large amounts of data within demanding time constraints. For instance, a mobile phone application will need to access and manipulate subscriber details in handling calls as well as in billing and user support. Search times that are proportional to the amount of data being searched are not acceptable in soft real-time systems. Lookup times not only have to be constant, but also have to be very fast!
One of the main composite data types used in programming is a collection of items (or elements, or objects). Erlang lists provide one way to implement a collection, but with more than a small number of items in the list, access to elements becomes slow. On average, we need to check through 50% of the elements in a collection to confirm that a given element is present, and we need to look at all the elements to verify that a given value is absent.
To handle fast searches, Erlang provides two mechanisms. This chapter introduces Erlang Term Storage (ETS) and Disk Erlang Term Storage (Dets), two mechanisms for memory- and disk-efficient storage and retrieval of large collections of data. Erlang also provides a full database application, Mnesia, which we cover in Chapter 13.
ETS tables store tuples, with access to the elements given through a key field in the tuple. The tables are implemented using hash tables and binary trees, with different representations providing different kinds of collections.