If a nonuniform random generator has a high-quality uniform
random number generator at its core, the main thing to test is whether the
generator output has the correct distribution. Fortunately, tiny coding
mistakes often result in egregious output errors, so simple tests may be
adequate to flush out bugs. However, some bugs are more subtle, and so
more sophisticated tests may be necessary. The recommendation is to start
with the simplest tests and work up to more advanced tests. The simplest
tests, besides being easiest to implement, are also the easiest to
understand. A software developer is more likely to respond well to being
told, “Looks like the average of your generator is 7 when it should be 8,”
than to being told, “I’m getting a small *p*-value from
my Kolmogorov-Smirnov test.”

If a probability distribution has a limited range, the simplest thing to test is whether the output values fall in that range. For example, an exponential distribution produces only positive values. If your test detects a single negative value, you’ve found a bug. However, for other distributions, such as the normal, there are no theoretical bounds on the outputs; all output values are possible, though some values are exceptionally unlikely.

There is one aspect of output ranges that cannot be tested effectively by black-box testing: boundary values. It may be impractical to test whether the endpoints of intervals are included. For example, suppose an RNG is expected ...

Start Free Trial

No credit card required