Optimal navigation in two-dimensional flows: control theory and reinforcement learning

Zermelo's navigation problem seeks the trajectory of minimal travel time between two points in a fluid flow. We address this problem for an agent -- such as a micro-robot or active particle -- that is advected by a two-dimensional flow, self-propels at a fixed speed smaller than or comparable to the characteristic flow velocity, and can steer its direction. The flows considered span increasing levels of complexity, from steady solid-body rotation to the Taylor-Green flow and fully developed turbulence in the inverse cascade regime. Although optimal control theory provides time-minimizing trajectories, these solutions become unstable in chaotic regimes realized for complex background flows. To design robust navigation strategies under such conditions, we apply reinforcement learning. Both action-value (Q-learning) and policy-gradient (one-step actor-critic) methods achieve successful navigation with comparable performance. Crucially, we show that agents trained on coarse-grained turbulent flows -- retaining only large-scale features -- generalize effectively to the full velocity field. This robustness to incomplete flow information is essential for practical navigation in real-world oceanic and atmospheric environments.