Game-Theoretic Foundations for Probability and Finance

11Calibrating Lookbacks and p‐Values

As we learned in Chapters 1 and 10, game‐theoretic probability provides a framework for testing. Skeptic tests Forecaster by trying to multiply the capital he risks by a large factor. In general, this testing is dynamic. Skeptic's degree of success – the factor by which he has multiplied his capital – varies over time. He might multiply it by a huge factor, apparently definitively discrediting Forecaster, but then lose most or all of his gains, casting this verdict into question. How can we insure against such a complete reversal? How can we make sure that when we look back and see the maximum capital that Skeptic has attained, we can claim at least some of that capital as evidence against Forecaster?

There is a simple way to retain at least a portion of Skeptic's gains. We can decide in advance on several target levels of capital at which we might stop betting, allocate our capital among these targets, and then imitate Skeptic with each of the resulting accounts until its target is reached. We call this a lookback trading strategy. As we will see in this chapter, this turns out to be the best we can do. We will apply the idea both to financial markets and to statistical testing.

To formalize the picture, we consider a game in which two players test Forecaster. We call them Skeptic and Rival Skeptic. Skeptic plays first, and Rival Skeptic can therefore imitate him with different accounts until different targets are reached. We develop this ...