Unclouding the sky of negatively curved manifolds

Let M be a complete simply connected Riemannian manifold, with sectional curvature K bounded above by -1. Under some assumptions on the geometry of the boundary of M, which are satisfied for instance if M is a symmetric space, or has dimension 2, we prove that given any family of horoballs in M, and any point x_0 outside these horoballs, it is possible to shrink uniformly, by a finite amount depending only on M, these horoballs so that some geodesic ray starting from x_0 avoids the shrunk horoballs. As an application, we give a uniform upper bound on the infimum of the heights of the closed geodesics in the finite volume quotients of M.