Search for the non-resonant production of Higgs boson pairs via gluon fusion and vector-boson fusion in the $b\bar{b}{\tau }^{+}{\tau }^{-}$ final state in proton-proton collisions at $\sqrt{s}=13$ TeV with the ATLAS detector

A search for the non-resonant production of Higgs boson pairs in the $HH\to b\bar{b}{\tau }^{+}{\tau }^{-}$ channel is performed using 140 fb${}^{-1}$ of proton-proton collisions at a centre-of-mass energy of $13$ TeV recorded by the ATLAS detector at the CERN Large Hadron Collider. The analysis strategy is optimised to probe anomalous values of the Higgs boson self-coupling modifier ${\kappa }_{\lambda }$ and of the quartic $HHVV$ ($V=W,Z$) coupling modifier ${\kappa }_{2V}$. No significant excess above the expected background from Standard Model processes is observed. An observed (expected) upper limit ${\mu }_{HH}<5.9$ $(3.3)$ is set at 95% confidence-level on the Higgs boson pair production cross-section normalised to its Standard Model prediction. The coupling modifiers are constrained to an observed (expected) 95% confidence interval of $-3.1<{\kappa }_{\lambda }<9.0$ ($-2.5<{\kappa }_{\lambda }<9.3$) and $-0.5<{\kappa }_{2V}<2.7$ ($-0.2<{\kappa }_{2V}<2.4$), assuming all other Higgs boson couplings are fixed to the Standard Model prediction. The results are also interpreted in the context of effective field theories via constraints on anomalous Higgs boson couplings and Higgs boson pair production cross-sections assuming different kinematic benchmark scenarios.