Quantum Temporal Fusion Transformer

The $\mathit{\text{Temporal}}\mathit{\text{Fusion}}\mathit{\text{Transformer}}$ (TFT), proposed by Lim $\mathit{\text{et}}\mathit{\text{al.}}$, published in $\mathit{\text{International}}\mathit{\text{Journal}}\mathit{\text{of}}\mathit{\text{Forecasting}}$ (2021), is a state-of-the-art attention-based deep neural network architecture specifically designed for multi-horizon time series forecasting. It has demonstrated significant performance improvements over existing benchmarks. In this work, we introduce the Quantum Temporal Fusion Transformer (QTFT), a quantum-enhanced hybrid quantum-classical architecture that extends the capabilities of the classical TFT framework. The core idea of this work is inspired by the foundation studies, $\mathit{\text{The}}\mathit{\text{Power}}\mathit{\text{of}}\mathit{\text{Quantum}}\mathit{\text{Neural}}\mathit{\text{Networks}}$ by Amira Abbas $\mathit{\text{et}}\mathit{\text{al.}}$ and $\mathit{\text{Quantum}}\mathit{\text{Vision}}\mathit{\text{Transformers}}$ by El Amine Cherrat $\mathit{\text{et}}\mathit{\text{al.}}$, published in $\mathit{\text{Nature}}\mathit{\text{Computational}}\mathit{\text{Science}}$ (2021) and $\mathit{\text{Quantum}}$ (2024), respectively. A key advantage of our approach lies in its foundation on a variational quantum algorithm, enabling implementation on current noisy intermediate-scale quantum (NISQ) devices without strict requirements on the number of qubits or circuit depth. Our results demonstrate that QTFT is successfully trained on the forecasting datasets and is capable of accurately predicting future values. In particular, our experimental results on two different datasets display that the model outperforms its classical counterpart in terms of both training and test loss. These results indicate the prospect of using quantum computing to boost deep learning architectures in complex machine learning tasks.