Near Optimality of Discrete-Time Approximations for Controlled McKean-Vlasov Diffusions and Interacting Particle Systems

We study stochastic optimal control problems for (possibly degenerate) McKean-Vlasov controlled diffusions and obtain discrete-time as well as finite interacting particle approximations. (i) Under mild assumptions, we first prove the existence of optimal relaxed controls by endowing the space of relaxed policies with a compact weak topology. (ii) Establishing continuity of the cost in control policy, we establish near-optimality of piecewise-constant strict policies, show that the discrete-time value functions (finite-horizon and discounted infinite-horizon) converge to their continuous-time counterparts as the timestep converges to zero, and that optimal discrete-time policies are near-optimal for the original continuous-time problem, where rates of convergence are also obtained. (iii) We then extend these approximation and near-optimality results to $N$-particle interacting systems under centralized or decentralized mean-field sharing information structure, proving that the discrete-time McKean-Vlasov policy is asymptotically optimal as $N\to \infty$ and the time step goes to zero. We thus develop an approximation of McKean-Vlasov optimal control problems via discrete-time McKean-Vlasov control problems (and associated numerical methods such as finite model approximation), and also show the near optimality of such approximate policy solutions for the $N$-agent interacting models under centralized and decentralized control.