Impala’s Place in the Big Data Ecosystem
The Cloudera Impala project arrives in the Big Data world at just the right moment. Data volume is growing fast, outstripping what can be realistically stored or processed on a single server. Some of the original practices for Big Data are evolving to open that field up to a larger audience of users and developers.
Impala brings a high degree of flexibility to the familiar database ETL process. You can query data that you already have in various standard Apache Hadoop file formats. You can access the same data with a combination of Impala, Apache Hive, and other Hadoop components such as Apache Pig or Cloudera search, without needing to duplicate or convert the data. When query speed is critical, the new Parquet columnar file format makes it simple to reorganize data for maximum performance of data warehouse-style queries.
Traditionally, Big Data processing has been like batch jobs from mainframe days, where unexpected or tough questions required running jobs overnight or all weekend. The goal of Impala is to express even complicated queries directly with familiar SQL syntax, running fast enough that you can get an answer to an unexpected question while a meeting or phone call is in progress. (We refer to this degree of responsiveness as “interactive.”)
For users and business intelligence tools that speak SQL, Impala brings a more effective development model than writing a new Java program to handle each new kind of analysis. Although the SQL ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access