Motivated by Dynamic Mode Decomposition algorithms, we provide lower bounds on the dimension of a finite-dimensional subspace F⊆L2(X) required for predicting the behavior of dynamical systems over long time horizons. We distinguish between two cases: (i) If F is determined by a finite partition of X we derive a lower bound that depends on the dynamical measure-theoretic entropy of the partition. (ii) We consider general finite-dimensional subspaces F and establish a lower bound for the dimension of F that is contingent on the spectral structure of the Koopman operator of the system, via the approximation entropy of F as studied by Voiculescu. Furthermore, we motivate the use of delay observables to improve the predictive qualities of Dynamic Mode Decomposition algorithms.