TensorFlow 1.x Deep Learning Cookbook

Getting ready

Let's see the expressions that we will need to make our RBM:

Forward pass: The information at visible units (V) is passed via weights (W) and biases (c) to the hidden units (h₀). The hidden unit may fire or not depending on the stochastic probability (σ is stochastic probability):

p(h_i|v₀) = σ(V^TW + c)_i

Backward pass: The hidden unit representation (h₀) is then passed back to the visible units through the same weights W, but different bias c, where they reconstruct the input. Again, the input is sampled:

p(v_i|h₀) = σ(W^Th₀ +b)_i

These two passes are repeated for k-steps or till the convergence is reached. According to researchers, k=1 gives good results, so we will keep k = 1.

The joint configuration of the visible vector V and ...