Hypernomics

CHAPTER 10An Amazon Mining Expedition

“Exploring the unknown requires tolerating uncertainty.”

Brian Greene

By now, we understand that to make sense of a part of the financial world, we need to gather as much helpful information about that market as possible. We will always want to know what customers value, so we must find information about useful features for any product or service realm we wish to examine. As always, markets will limit their purchases, so we need data about quantities sold. Occasionally, there are constraints that some markets face, and if they exist, we need to be aware of them. In essence, we know what we need. Now, where do we find it?

While a wide variety of data services describe many markets for one‐time or annual fees, we may often find ourselves interested in a particular part of the economy but have yet to receive data instantly retrievable about it. If we have no available market data to buy, we could throw our hands up and come up empty or get to work. Since we are interested in the knowledge that such work will bring us, we decide to dig in and get the information and insight we need. In those cases, we must mine, organize, and analyze our information. This is data mining.

As of this writing, in the early 21st century, we have a large retailer, Amazon,^1,2 that sells all manner of products via their website and then ships to its customers. While Amazon does not put up large data sets in the formats we know we have to create, it offers a fantastic ...