Transfer operator for the Gauss' continued fraction map. I. Structure of the eigenvalues and trace formulas

Let L be the transfer operator associated with the Gauss’ continued fraction map, known also as the Gauss-Kuzmin-Wirsing operator, acting on the Banach space. In this work we prove a two-term asymptotic formula for the eigenvalues of L, show their algebraic simplicity, sign alternation pattern, and decrease in absolute value. This settles, in a stronger form, the conjectures of D. Mayer and G. Roepstorff (1988), A.J. MacLeod (1992), Ph. Flajolet and B. Vallee (1995), also supported by several other authors. Further, we find an exact series for the eigenvalues, which also gives the canonical decomposition of trace formulas due to D. Mayer (1976) and K.I. Babenko (1978). This crystallizes the contribution of each individual eigenvalue in the trace formulas.