A spectral extremal problem on non-bipartite triangle-free graphs

A theorem of Nosal and Nikiforov states that if $G$ is a triangle-free graph with $m$ edges, then $\lambda (G)\le \sqrt{m}$, where the equality holds if and only if $G$ is a complete bipartite graph. A well-known spectral conjecture of Bollobás and Nikiforov [J. Combin. Theory Ser. B 97 (2007)] asserts that if $G$ is a $K_{r+1}$-free graph with $m$ edges, then ${\lambda }_1^2(G)+{\lambda }_2^2(G)\le (1-\frac{1}{r})2m$. Recently, Lin, Ning and Wu [Combin. Probab. Comput. 30 (2021)] confirmed the conjecture in the case $r=2$. Using this base case, they proved further that $\lambda (G)\le \sqrt{m-1}$ for every non-bipartite triangle-free graph $G$, with equality if and only if $m=5$ and $G=C_5$. Moreover, Zhai and Shu [Discrete Math. 345 (2022)] presented an improvement by showing $\lambda (G)\le \beta (m)$, where $\beta (m)$ is the largest root of $Z(x):=x^3-x^2-(m-2)x+m-3$. The equality in Zhai–Shu’s result holds only if $m$ is odd and $G$ is obtained from the complete bipartite graph $K_{2,\frac{m-1}{2}}$ by subdividing exactly one edge. Motivated by this observation, Zhai and Shu proposed a question to find a sharp bound when $m$ is even. We shall solve this question by using a different method and characterize three kinds of spectral extremal graphs over all triangle-free non-bipartite graphs with even size. Our proof technique is mainly based on applying Cauchy interlacing theorem of eigenvalues of a graph, and with the aid of a triangle counting lemma in terms of both eigenvalues and the size of a graph.