TensorFlow 1.x Deep Learning Cookbook

How to do it...

We proceed with the recipe as follows:

1. Consider this piece of code which runs a matrix multiplication on a single GPU.

# single GPU (baseline)import tensorflow as tf# place the initial data on the cpuwith tf.device('/cpu:0'):  input_data = tf.Variable([[1., 2., 3.],    [4., 5., 6.],    [7., 8., 9.],    [10., 11., 12.]])b = tf.Variable([[1.], [1.], [2.]]) # compute the result on the 0th gpuwith tf.device('/gpu:0'):  output = tf.matmul(input_data, b) # create a session and runwith tf.Session() as sess:  sess.run(tf.global_variables_initializer())  print sess.run(output)

2. Partition the code with in graph replication as in the following snippet between 2 different GPUs. Note that the CPU is acting as the master node distributing the graph ...