Information Theory Meets Power Laws

9Power Laws for Repetitions

In the previous chapter, we have analyzed power laws connected to the growth of block mutual information (1.132). We have seen that these laws arise naturally if the persistent topic of a random text is infinitely complex. Equivalently, we may say that in this case, there exists an infinitely complex world that is repeatedly described by the random text. In contrast, in this chapter, we would like to study the power‐law logarithmic growth of maximal repetition (1.133), which also seems to arise in texts created by humans, as introduced in Section 1.13.

The power‐law logarithmic growth of maximal repetition was discovered by Dębowski (2012b, 2015b) accidentally. The initial motivation for this research was confirming a hypothesis of Dębowski (2011) that the maximal repetition for natural language grows logarithmically, which was connected to the theorem about facts and words originally stated for minimal grammar‐based codes, see Problem 7.4. A converse statement turned out to be likely true for natural language. Consequently, we have asked the question what kind of stochastic processes can exhibit a power‐law logarithmic growth of maximal repetition and in what ways they can resemble natural language. This idea inspired further papers (Dębowski, 2017, 2018b), where we constructed random hierarchical association (RHA) processes and demonstrated some general bounds for the maximal repetition in terms of conditional Rényi entropy rates.

In fact, the ...