Data Warehousing with Greenplum

Chapter 7. Integrating with Real-Time Response

John Knapp

GemFire-Greenplum Connector

We designed Greenplum to provide analytic insights into large amounts of data. We did not design it for real-time response. Yet, many real-world problems require a system that does both. At Pivotal, we use GemFire for real-time requirements and the GemFire-Greenplum Connector to integrate the two.

Problem Scenario: Fraud Detection

As more businesses interact with their customers digitally, ensuring trustworthiness takes on a critical role. More than 17 million Americans were victims of identity theft in 2014, the latest year for which statistics are available. Fraudulent transactions stemming from identity theft—fraudulent credit card purchases, insurance claims, tax refunds, telecom services, and so on—cost businesses and consumers more than $15 billion that year, according to the Department of Justice’s Bureau of Justice Statistics.

Detecting and stopping fraudulent transactions related to identity theft is a top priority for many banks, credit card companies, insurers, tax authorities, as well as digital businesses across a variety of industries. Building these systems typically relies on a multistep process, including the difficult steps of moving data in multiple formats between analytical systems, which are used to build and run predictive models, and transactional systems, where the incoming transactions are scored for the likelihood of fraud. Analytical systems and transactional systems ...