Effect of molecular hydrogen self-shielding modeling on early Reionization Era galaxies in radiative hydrodynamic cosmological simulations

Accurately modeling molecular hydrogen ($\text{H}_{2}$) is an important task in cosmological simulations because it regulates star formation. One fundamental property of $\text{H}_{2}$ is the ability to self-shield, a phenomenon in which the $\text{H}_{2}$ in the outer layer of a molecular cloud absorbs the photodissociating Lyman-Werner UV radiation and shields the inner $\text{H}_{2}$. Historically, numerical approximations have been utilized to avoid intensive ray-tracing calculations. This paper evaluates the use of the Sobolev-like density-gradient approximation in $\text{H}_{2}$ self-shielding modeling and tests its agreement with a more rigorous adaptive ray-tracing method in cosmological simulations. We ran four high-resolution zoom-in cosmological simulations to investigate the models' effects in the early Reionization Era ($z \geq 12$). We find that the approximation model returns a higher $\text{H}_{2}$ photodissociation rate in low gas density environments but a lower rate when gas density is high, resulting in low-mass halos having less $\text{H}_{2}$ while high-mass halos having more $\text{H}_{2}$. The approximation also hinders star formation in small halos, but it less affects the stellar mass of larger halos. Inside a halo, the discrepancies between the two models regarding $\text{H}_{2}$ fraction, temperature, and stellar mass are radially dependent. On a large scale, the simulations using the approximation have less $\text{H}_{2}$ in the intergalactic medium and may experience a slower reionization process. These results show that the Sobolev-like approximation alters properties of galaxies and the large-scale universe when compared to the ray-tracing treatment, emphasizing a need for caution when interpreting results from these two techniques in cosmological simulations.