Summary
This concludes our Appendix where we covered some topics based on performance tuning, sizing up your executors, handling data skew, configuring security, setting up a Jupyter notebook with Spark and finally broadcast variables and accumulators.
There are many more topics still to be covered, but we hope that this book has given you an effective quick-start with Spark 2.0, and you can use it to explore Spark further. Of course, Spark is one of the fastest moving projects out there, so by the time the book is out there will surely be many new features. One of the best places to keep up-to-date on the latest changes is http://spark.apache.org/documentation.html, where you can see the list of releases and the latest news.
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access