Learning about TensorFlowOnSpark
In the year 2016, Yahoo open sourced TensorFlowOnSpark, a Python framework for performing TensorFlow-based distributed deep learning on Spark clusters. Since then, it has undergone a lot of developmental changes and is one of the most active repositories regarding the distributed deep learning framework.
The TensorFlowOnSpark (TFoS) framework allows you to run distributed TensorFlow applications from within Spark programs. It runs on the existing Spark and Hadoop clusters. It can use existing Spark libraries such as SparkSQL or MLlib (the Spark machine learning library).
TFoS is automatic, so we do not need to define the nodes as PS nodes, nor do we need to upload the same code to all of the nodes in the cluster. By ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access