Dual-mode ground-state cooling in quadratic optomechanical systems: from multistability to general dark-mode suppression

We theoretically investigate a quadratic optomechanical system comprising a single-mode optical cavity linearly coupled to one mechanical resonator and quadratically coupled to a second resonator. By tuning the cavity detuning and optomechanical coupling strengths, we demonstrate the transition from optical bistability to multistability with up to seven steady-state solutions. Notably, simultaneous ground-state cooling of both mechanical resonators occurs on the dynamically stable branch of the nonlinear steady-state solutions, offering new opportunities for combined nonlinear optical and quantum cooling functionalities. Beyond the multistable regime, we systematically study dual-mode ground-state cooling and find that robust simultaneous cooling can be achieved over a broad parameter range, except when the linear and quadratic couplings become comparable, where a dark-mode effect arises. In this case, tuning the second-order optomechanical-induced frequency shifts effectively suppresses dark-mode interference, enabling controllable and simultaneous ground-state cooling. Our results provide a versatile framework for engineering multimode quantum states in optomechanical systems and open new avenues for the development of multifunctional quantum devices, including ultra-sensitive sensors, scalable quantum memories, and integrated quantum networks.