Ground state solution for a generalized Choquard Schrodinger equation with vanishing potential in homogeneous fractional Musielak Sobolev spaces

This paper aims to establish the existence of a weak solution for the following problem: \begin{equation*} (-Δ)^{s}{$\mathcal{H}$}u(x) +V(x)h(x,x,|u|)u(x)=\left(\int{$\mathbb{R}$^{N}}\dfrac{K(y)F(u(y))}{|x-y|^λ}dy \right) K(x)f(u(x)) \ \hbox{in} \ $\mathbb{R}$^{N}, \end{equation*} where $N\ge 1$, $s\in (0,1),\lambda \in (0,N),\mathcal{H}(x,y,t)={\int }_0^{\mathrm{\mid }t\mathrm{\mid }}h(x,y,r)rdr,$ $h:{\mathbb{R}}^N\times {\mathbb{R}}^N\times [0,\mathrm{\infty })\to [0,\mathrm{\infty })$ is a generalized $N$-function and $(-\mathrm{\Delta }{)}_{\mathcal{H}}^s$ is a generalized fractional Laplace operator. The functions $V,K:{\mathbb{R}}^N\to (0,\mathrm{\infty })$, non-linear function $f:\mathbb{R}\to \mathbb{R}$ are continuous and $F(t)={\int }_0^{t}f(r)dr\mathrm{.}$ First, we introduce the homogeneous fractional Musielak-Sobolev space and investigate their properties. After that, we pose the given problem in that space. To establish our existence results, we prove and use the suitable version of Hardy-Littlewood-Sobolev inequality for Lebesque Musielak spaces together with variational technique based on the mountain pass theorem. We also prove the existence of a ground state solution by the method of Nehari manifold.