Summary
In this chapter, we have covered the basics of ELT, and Spark's ability to interact with a variety of sources including standard text, CSV, TSV, and JSON files. We moved on to look at accessing filesystems including local filesystems, HDFS, and S3. Finally, we spent some time on helping you understand access to a variety of NoSQL databases and the connectors available. As you can see, we have covered a few of the popular systems, but the massive open-source ecosystem around Spark means there are new connectors coming almost on a monthly basis. It is highly recommended to look closely at the project's GitHub page for the latest developments.
We'll now move on to the next chapter, where we are going to focus on Spark SQL, DataFrames, and ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access