Confidence Intervals Using Turing's Estimator: Simulations and Applications

Turing’s estimator allows one to estimate the probabilities of outcomes that either do not appear or only rarely appear in a given random sample. We perform a simulation study to understand the finite sample performance of several related confidence intervals (CIs) and introduce an approach for selecting the appropriate CI for a given sample. We give an application to the problem of authorship attribution and apply it to a dataset comprised of tweets from users on X (Twitter). Further, we derive several theoretical results about asymptotic normality and asymptotic Poissonity of Turing’s estimator for two important discrete distributions.