Big Data Computing

353Economics of Big Data

We will use the following cost estimate methodology. We will randomly

select approximately 70,000 domain names, crawl the home page of each,

and extract all images from the crawled pages. Next, we will randomly select

100,000 images from the gathered set. Then, we will retrieve and process

selected images as described above. From the costs determined in this exer-

cise, we can extrapolate the cost per billion.

First, let us look at numbers associated with computing and bandwidth

capacity. Low-end EC2 instances can successfully retrieve and process about

100,000 images per hour. Once processed, the resulting image data within

the catalog averages to approximately 2 k bytes compressed. Further, we nd

that it ta ...