Fuzzy Data Matching with SQL

Preface

This book contains the patterns, practices, techniques, and tricks I’ve picked up over the decades, usually around the problem of “Is this list of data related to anything in that table?” The canonical example is a cold-call list pulled from…“somewhere” (we’ll be kind), and now Marketing (it’s always Marketing) wants you to match this list they probably paid for against the company’s existing customer database. Why? There are lots of reasons, but these are the chief two:

Identify new prospects

Filter out existing customers from the list and send the new prospects down a low-cost, standardized route with perhaps a cold call, mailing list, etc. Think Glengarry Glen Ross.

Upsell and cross-sell existing customers

Filter out new prospects, and if an existing customer shows up at a trade show and expresses interest, perhaps they’re not aware of your entire product offering or haven’t been “touched” in a while to determine their needs. Get a salesperson in front of them with some incentives, pronto.

There are other reasons, of course. Two companies merging and wanting to combine their customer relationship management (CRM) systems and eliminate duplicates is another common reason. Often the same types of techniques are used to de-duplicate even within a single dataset, like a CRM database. Or when someone comes to you with data found on the “dark web” attributed to your company. Is it yours? Is it complete? Accurate? Current?

Don’t deal with data about humans? The second ...