Nonconventional limit theorems in discrete and continuous time via martingales

We obtain functional central limit theorems for both discrete time expressions of the form $1/\sqrt{N}\sum_{n=1}^{[Nt]}(F(X(q_1(n)),\ldots, X(q_{\ell}(n)))-\bar{F})$ and similar expressions in the continuous time where the sum is replaced by an integral. Here $X(n),n\geq0$ is a sufficiently fast mixing vector process with some moment conditions and stationarity properties, $F$ is a continuous function with polynomial growth and certain regularity properties, $\bar{F}=\int F\,d(\mu\times\cdots\times\mu)$, $\mu$ is the distribution of $X(0)$ and $q_i(n)=in$ for $i\le k\leq\ell$ while for $i>k$ they are positive functions taking on integer values on integers with some growth conditions which are satisfied, for instance, when $q_i$'s are polynomials of increasing degrees. These results decisively generalize [Probab. Theory Related Fields 148 (2010) 71-106], whose method was only applicable to the case $k=2$ under substantially more restrictive moment and mixing conditions and which could not be extended to convergence of processes and to the corresponding continuous time case. As in [Probab. Theory Related Fields 148 (2010) 71-106], our results hold true when $X_i(n)=T^nf_i$, where $T$ is a mixing subshift of finite type, a hyperbolic diffeomorphism or an expanding transformation taken with a Gibbs invariant measure, as well as in the case when $X_i(n)=f_i({\Upsilon }_n)$, where ${\Upsilon }_n$ is a Markov chain satisfying the Doeblin condition considered as a stationary process with respect to its invariant measure. Moreover, our relaxed mixing conditions yield applications to other types of dynamical systems and Markov processes, for instance, where a spectral gap can be established. The continuous time version holds true when, for instance, $X_i(t)=f_i(\xi_t)$, where $\xi_t$ is a nondegenerate continuous time Markov chain with a finite state space or a nondegenerate diffusion on a compact manifold. A partial motivation for such limit theorems is due to a series of papers dealing with nonconventional ergodic averages.