Chapter 6. Apache Sedona and the PyData Ecosystem
Python is an all-purpose, interpreted language with high-level abstractions. It is known for its simplicity and readability to the extent that perfectly clean code written in Python has been nicknamed Pythonic. Fast prototyping, ease of use, and a rich set of community-driven frameworks and libraries made Python very popular among those working in data. Initially, in the Hadoop era, data engineers used Java and Scala to process large amounts of data, while data scientists mainly used R to create machine learning models. However, Python was so convenient to use that, in both areas, it has become great for both data engineering and data science. Despite the native slowness of Python and compute-heavy resource utilization, the Python community created integrations with low-level fast languages like C, Fortran, Java, and recently, Rust.
Up to 2019, Apache Sedona only exposed its API for Scala, Java, and SQL. Thanks to a community-driven initiative, Apache Sedona got initial support for the DataFrame API in Python. In early 2020, the missing spatial RDD operations were also included, and the library was officially published in the PyPI repositories, making it publicly available and easy to install. The Apache Sedona team initially anticipated thousands of downloads per month, but expectations were exceeded, and today, the project is standing at over 1.5 million monthly downloads.
The complexity of geospatial problems requires using ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access