Lower tails via relative entropy

We show that the naive mean-field approximation correctly predicts the leading term of the logarithmic lower tail probabilities for the number of copies of a given subgraph in $G(n,p)$ and of arithmetic progressions of a given length in random subsets of the integers in the entire range of densities where the mean-field approximation is viable. Our main technical result provides sufficient conditions on the maximum degrees of a uniform hypergraph $\mathcal{H}$ that guarantee that the logarithmic lower tail probabilities for the number of edges induced by a binomial random subset of the vertices of $\mathcal{H}$ can be well-approximated by considering only product distributions. This may be interpreted as a weak, probabilistic version of the hypergraph container lemma that is applicable to all sparser-than-average (and not only independent) sets.