Success Probability Estimation with Applications to Clinical Trials

CHAPTER 2

REPRODUCIBILITY PROBABILITY ESTIMATION

Scientific Method hinges on the reproducibility of experimental outcomes.

Clinical trials data are analyzed through statistical tests and, consequently, the reproducibility of their outcomes becomes relevant. Nevertheless, the outcomes of statistical tests, besides belonging to the set: {significant, non-significant}, are random ones because they depend on the sample(s) selected randomly and/or on the randomized allocation of the treatments. Hence, even the reproducibility of the test result inherits this randomness, and it should be evaluated in terms of probability of reproducing a certain outcome of the statistical test itself.

It is of note that a significant outcome represents experimental proof that the new drug is effective, whereas a non-significant one implies that nothing is proved. So, the quantity of main interest is the reproducibility probability of a statistically significant outcome in a new experiment, whose settings are identical to those of the old one. Nevertheless, since a non-significant result is not experimental proof of the null hypothesis and it may represent a type II error, it is assumed that the reproducibility probability (RP) is the probability of finding a significant outcome in a second experiment (with identical settings) independently from the outcome of the first one. RP is strictly related to success probability SP, and it is, indeed, a particular case of the SP.

In this Chapter, the estimation ...