Reconstruction of Stochastic Dynamics from Large Streamed Datasets

The complex dynamics of physical systems can often be modeled with stochastic differential equations. However, computational constraints inhibit the estimation of dynamics from large time-series datasets. I present a method for estimating drift and diffusion functions from inordinately large datasets through the use of incremental, online, updating statistics. I demonstrate the validity and utility of this method by analyzing three large, varied synthetic datasets, as well as an empirical turbulence dataset. This method will hopefully facilitate the analysis of complex systems from exceedingly large, “big data” scientific datasets, as well as real-time streamed data.