Upper tails via high moments and entropic stability

Suppose that $X$ is a bounded-degree polynomial with nonnegative coefficients on the $p$-biased discrete hypercube. Our main result gives sharp estimates on the logarithmic upper tail probability of $X$ whenever an associated extremal problem satisfies a certain entropic stability property. We apply this result to solve two long-standing open problems in probabilistic combinatorics: the upper tail problem for the number of arithmetic progressions of a fixed length in the $p$-random subset of the integers and the upper tail problem for the number of cliques of a fixed size in the random graph $G_{n,p}$. We also make significant progress on the upper tail problem for the number of copies of a fixed regular graph $H$ in $G_{n,p}$. To accommodate readers who are interested in learning the basic method, we include a short, self-contained solution to the upper tail problem for the number of triangles in $G_{n,p}$ for all $p=p(n)$ satisfying $n^{-1}\log n\ll p \ll 1$.