The Big Data Transformation

Chapter 6. Whatever You Do, Don’t Do This, Warns Etsy

Up to this point, we’ve spent the bulk of this document talking about—and illustrating—real-world best practices for integrating an analytical database like Vertica into your data processing environment. Now we’re going to take an opposite approach: we’re going to tell you what not to do—lessons from experts on how to avoid serious mistakes when implementing a big-data analytics database.

Don’t Forget to Consider Your End User When Designing Your Analytics System

“That is the most important thing that will drive the tools you choose,” said Chris Bohn, “CB,” a senior database engineer with Etsy, a marketplace where millions of people around the world connect, both online and offline, to make, sell, and buy unique goods. Etsy was founded in 2005 and is headquartered in Brooklyn, New York.

Etsy uses HPE Vertica to analyze a 130 TB database to discover new revenue opportunities. To improve performance by an order of magnitude, Etsy replaced its PostgreSQL system with HPE Vertica to efficiently and quickly analyze more than 130 TB of data. Bohn says that the greatest benefits are accessibility and speed, such that use of the tool has spread to all departments. “Queries that previously took many days to run now run in minutes,” says Bohn. This has increased companywide productivity.

But Etsy considered the end users of the analytics database before choosing Vertica—and those end users, it turned out, were mainly analysts.

Analysts ...