An ESO/VLT survey of NIR (Z<=25) selected galaxies at redshifts 4.5<z<6: constraining the cosmic star formation rate near the reionization epoch

We present the results of a VLT and HST imaging survey aimed at the identification of 4.5<z<6 galaxies. In the VLT data, a set of broad and intermediate band filters has been used to select 13 high–$z$ candidates in a Z(AB)<25 catalog, over an area of about 30 arcmin2. Discrimination against lower redshift interlopers (mainly early–type galaxies at high redshift and cool Galactic stars) has been done combining morphological and spectral classification. This sample has been combined with a deeper I<27.2 sample obtained from the Hubble Deep Field campaigns. The VLT final sample consists of 13 high z candidates, 4 of which are identified with high confidence as z>4.5 galaxies. The resulting integral surface density of the Z<25 candidates at z>4.5z is in the range 0.13-0.44/arcmin^2 and that in the highest redshift bin 5<z<6 is between 0.07-0.13/arcmin^2. In the two HDFs, we identify at I<27.2 25 galaxies in the range 4.5<z<5 and 16 at 5<z<6$, corresponding to surface densities of 3.1 arcmin^-2 and 2 arcmin^-2, respectively. We show that the observed Z<25 UV luminosity density appear to drop by about one order of magnitude from z~3 to z~6. However, if we apply a threshold to obtain an absolute–magnitude limited sample, the UV luminosity density results to be roughly constant up to z~6. We finally show that recent semi-analytic hierarchical models for galaxy formation, while predicting a nearly constant total UV luminosity density up to z~6, under-predict the observed UV luminosity density at Z<25 and over-predicts the I<27.2 one. This behaviour can be understood in term of a poor match to the slope of the UV luminosity function.