Gravitational-Wave Signatures of Highly Eccentric Stellar-Mass Binary Black Holes in Galactic Nuclei

A significant fraction of compact-object mergers in galactic nuclei are expected to be eccentric in the Laser Interferometer Space Antenna (LISA) frequency sensitivity range, $10^{-4} - 10^{-1}\ \rm Hz$. Several compact binaries detected by the LIGO-Virgo-KAGRA Collaboration may retain hints of residual eccentricity at $\sim 10$ Hz, suggesting dynamical or triple origins for a significant fraction of the gravitational-wave-observable population. In triple systems, von-Zeipel-Lidov-Kozai oscillations perturb both the eccentricity and the argument of pericentre, $\omega$, of the inner black hole binary. The latter could be fully \textit{circulating}, where $\omega$ cycles through $2\pi$, or may \textit{librate}, with $\omega$ ranges about a fixed value with small or large variation. We use \texttt{TSUNAMI}, a regularised N-body code with up to 3.5 post-Newtonian (PN) term corrections, to identify four different families of orbits: (i) circulating, (ii) small and (iii) large amplitude librating, and (iv) merging orbits. We develop and demonstrate a new method to construct gravitational waveforms using the quadrupole formula utilising the instantaneous {\it total} acceleration of each binary component in \texttt{TSUNAMI}. We show that the four orbital families have distinct waveform phenomenologies, enabling them to be distinguished if observed in LISA. The orbits are also distinguishable from an isolated binary or from a binary perturbed by a different tertiary orbit, even if the secular timescale is the same. Future burst timing models will be able to distinguish the different orbital configurations. For efficient binary formation, about $\sim 1000$ binaries can have highly eccentric, librating orbits in the Galactic Centre.