Complexity of the Graph Homomorphism Problem w.r.t. Degeneracy

The graph homomorphism problem HOM is: given an $n$-vertex source graph $G$ and an $h$-vertex target graph $H$, is there a mapping from $V(G)$ to $V(H)$ that preserves edges? A straightforward brute-force algorithm for HOM has running time $O(2^{n \log h})$ and it is known that, under ETH, there are no $2^{o(n \log h)}$ algorithms. In recent years, less restrictive graph parameters $p$ have been identified that allow one to solve HOM in time $p(H)^{O(n)}$. Examples include treewidth, maximum degree, and track number. On the other hand, it is known that the chromatic number parameter is too small: under ETH, HOM cannot be solved in time $\chi(H)^{O(n)}$. We study the complexity of HOM in terms of the degeneracy of $H$. This is perhaps the most natural unresolved graph parameter between the known algorithmic and hardness regimes: on the one hand, each of bounded treewidth, bounded maximum degree, and bounded track number implies bounded degeneracy; on the other hand, bounded degeneracy implies bounded chromatic number. Our results show that, at the same time, the influence of degeneracy of $H$ on the complexity of HOM differs significantly from that of the previously studied parameters. We show that, under ETH, there is no $2^{o(degen(H) n)}$ algorithm for any value of $degen(H)$ as a function of $n$. We also show that bounded degeneracy alone does not make target size benign: even targets with $degen(H)$ at most $2$ and quasi-polynomial size force $n^{\Omega(n)}$-scale hardness. Finally, we introduce a no-compression barrier that explains why the known fine-grained lower bounds for sparse $2$-CSP are not tight under ETH. Moreover, it shows that substantially stronger lower bounds for polynomial-target degeneracy are unlikely to follow from standard reductions from sparse $3$-SAT.