Mastering Machine Learning Algorithms

Direct sampling

With direct sampling, our goal is to approximate the full joint probability through a sequence of samples drawn from each conditional distribution. If we assume that the graph is well-structured (without unnecessary edges) and we have N variables, the algorithm is made up of the following steps:

1. Initialize the variable N_Samples.
2. Initialize a vector S with shape (N, N_Samples).
3. Initialize a frequency vector F_Samples with shape (N, N_Samples). In Python, it's better to employ a dictionary where the key is a combination (x₁, x₂, x₃, ..., x_N).
4. For t=1 to N_Samples:
   1. For i=1 to N:
      1. Sample from P(X_i|Predecessors(X_i))
      2. Store the sample in S[i, t]
   2. If F_Samples contains the sampled tuple S[:, t]:
      1. F_Samples[S[:, t]] += 1
   3. Else:
      1. F_{Samples ...}