A version of Herbert A. Simon's model with slowly fading memory and its connections to branching processes

Construct recursively a long string of words w1. … wn, such that at each step k, w k+1 is a new word with a fixed probability p $\in$ (0, 1), and repeats some preceding word with complementary probability 1 – p. More precisely, given a repetition occurs, w k+1 repeats the j-th word with probability proportional to j $\alpha$ for j = 1,. … , k. We show that the proportion of distinct words occurring exactly times converges as the length n of the string goes to infinity to some probability mass function in the variable $\ge$ 1, whose tail decays as a power function when 1 – p > $\alpha$/(1 + $\alpha$), and exponentially fast when 1 – p < $\alpha$/(1 + $\alpha$).