The use of time series for sequential online prediction (SOP) has long been a research topic, but achieving robust and computationally efficient SOP with non-stationary time series remains a challenge. This paper reviews a framework, called Bayesian Dynamic Ensemble of Multiple Models (BDEMM), which addresses SOP in a theoretically elegant way, and have found widespread use in various fields. BDEMM utilizes a model pool of weighted candidate models, adapted online using Bayesian formalism to capture possible temporal evolutions of the data. This review comprehensively describes BDEMM from five perspectives: its theoretical foundations, algorithms, practical applications, connections to other research, and strengths, limitations, and potential future directions.