R Deep Learning Essentials - Second Edition

Random search

An alternative approach to hyper-parameter selection is searching through random sampling. Rather than pre­-specifying all of the values to try and create all possible combinations, one can randomly sample values for the parameters, fit a model, store the results, and repeat. To get a very large sample size, this too would be computationally demanding, but you can specify just how many different models you are willing to run. Therefore this approach gives you a spread over the combination of hyper-parameters.

For random sampling, all that need to be specified are values to randomly sample, or distributions to randomly draw from. Typically, some limits would also be set. For example, although a model could theoretically have ...