February 2024
Intermediate to advanced
198 pages
4h 41m
English
Content preview from Hands-On Entity Resolution
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Chapter 9. Cloud Entity Resolution Services
In the last chapter, we saw how to scale up our entity resolution process to run on a Google Cloud–managed Spark cluster. This approach allowed us to match larger datasets in a reasonable time but it required us to do quite a bit of setup and management ourselves.
An alternative approach is to use entity resolution API provided by a cloud provider to perform the hard work for us. Google, Amazon, and Microsoft all offer these services.
In this chapter, we will use the entity reconciliation service, provided as part of Google’s Enterprise Knowledge Graph API, to resolve the MCA and Companies House datasets we examined in Chapters 6 and 8. We will:
- Upload our standardized datasets to Google’s data warehouse, BigQuery.
- Provide a mapping of our data schema to a standard ontology.
- Invoke the API from the console (we will also invoke the API using a Python script).
- Use some basic SQL to process the results.
To complete the chapter we will examine how well the service performs.
Introduction to BigQuery
BigQuery is Google’s fully managed, serverless data warehouse that enables scalable analysis over petabytes of data. It is a platform as a service that supports data querying and analysis using a dialect of SQL.
To begin, we select the BigQuery product from the Google Cloud console. Under ANALYSIS we select “SQL workspace.”
Our first step is to select “Create dataset” from the ellipsis menu alongside your project name, as shown in Figure 9-1 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.