Learning PySpark

TensorFrames – quick start

After all this preamble, let's jump start our use of TensorFrames with this quick start tutorial. You can download and use the full notebook within Databricks Community Edition at http://bit.ly/2hwGyuC.

You can also run this from the PySpark shell (or other Spark environments), like any other Spark package:

# The version we're using in this notebook
$SPARK_HOME/bin/pyspark --packages tjhunter:tensorframes:0.2.2-s_2.10  

# Or use the latest version 
$SPARK_HOME/bin/pyspark --packages databricks:tensorframes:0.2.3-s_2.10

Note, you will only use one of the above commands (that is, not both). For more information, please refer to the databricks/tensorframes GitHub repository (https://github.com/databricks/tensorframes).

Configuration ...