Mastering Spark for Data Science

GIS lookup

In the previous section, we were covering an interesting use case, how to extract location entities from unstructured data. In this section, we will make our enrichment process even smarter by trying to retrieve the actual geographical coordinate information (such as latitude and longitude) based on the locations of entities we were able to identify. Given an input string London, can we detect the city of London - UK together with its relative latitude and longitude? We will be discussing how to build an efficient geo lookup system that does not rely on any external API and which can process location data of any scale by leveraging the Spark framework and the Reduce-Side-Join pattern. When building this lookup service, we will have ...