Data Analysis with Python and PySpark

Appendix C. Some useful Python concepts

Python is a fascinating and complex language. While there is a trove of beginners guides to learn the basics of the language, some new or more complex aspects of Python are less discussed. This appendix is a nonexhaustive compendium of intermediate to advanced Python concepts that will be useful when working with PySpark.

C.1 List comprehensions

List comprehensions are one of Python’s constructs that, once you understand, you’ll wonder how you were able to code without using them. At the core, they are nothing more than iterations over a list. Their power comes from their conciseness and how readable they are. We start using them in chapter 4 when providing multiple columns to methods like select() and ...