Optimal Complex-Valued Prototype Filter Design for GFDM Systems

One of the main challenges with generalized frequency division multiplexing (GFDM) systems is prototype filter design. A poorly designed filter increases inherent and out-of-band (OOB) interferences. In this paper, we introduce a novel optimal prototype filter for GFDM systems that eliminates the negative effects of intrinsic interferences. We introduce a complex-valued pulse shape similar to a single-sideband (SSB) modulation scheme, which significantly improves bandwidth efficiency. Specifically, we introduce an optimization problem to design an optimal pulse shape filter to reduce all intrinsic interference to zero. We derive analytical expressions to evaluate the bit error rate (BER) of the system and show how the designed optimal prototype filter outperforms its current counterparts.