Related papers: System Identification with Copula Entropy

Recently, the concept of cumulative residual entropy (CRE) has been studied by many researchers in higher dimensions. In this article, we extend the definition of (dynamic) cumulative past entropy (DCPE), a dual measure of (dynamic) CRE, to…

We describe a systematic development of kinetic entropy as a diagnostic in fully kinetic particle-in-cell (PIC) simulations and use it to interpret plasma physics processes in heliospheric, planetary, and astrophysical systems. First, we…

A frequent task in exploratory data analysis consists in examining pairwise dependencies between data variables. Popular approaches include visualizing correlation or scatter plot matrices. However, both methods can be misleading. The…

The determination of liquid phase equilibria plays an important role in chemical process simulation. This work presents a generalization of an approach called the convex envelope method (CEM), which constructs all liquid phase equilibria…

Robust learning in the presence of non-Gaussian and statistically dependent noise remains a fundamental challenge in signal processing and adaptive systems. Although information-theoretic learning criteria such as correntropy offer strong…

Sample selection improves the efficiency and effectiveness of machine learning models by providing informative and representative samples. Typically, samples can be modeled as a sample graph, where nodes are samples and edges represent…

A framework named Copula Component Analysis (CCA) for blind source separation is proposed as a generalization of Independent Component Analysis (ICA). It differs from ICA which assumes independence of sources that the underlying components…

Cellular automata (CA) exemplify systems where simple local interaction rules can lead to intricate and complex emergent phenomena at large scales. The various types of dynamical behavior of CA are usually categorized empirically into…

The cross-entropy method (CE) developed by R. Rubinstein is an elegant practical principle for simulating rare events. The method approximates the probability of the rare event by means of a family of probabilistic models. The method has…

Discovering nonlinear differential equations that describe system dynamics from empirical data is a fundamental challenge in contemporary science. Here, we propose a methodology to identify dynamical laws by integrating denoising techniques…

Probabilistic forecasts in the form of ensemble of scenarios are required for complex decision making processes. Ensemble forecasting systems provide such products but the spatio-temporal structures of the forecast uncertainty is lost when…

Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanics approach to analyze sparse equation discovery algorithms, which typically balance data fit and…

Entropy is a fundamental thermodynamic quantity indicative of the accessible degrees of freedom in a system. While it has been suggested that the entropy of a mesoscopic system can yield nontrivial information on emergence of exotic states,…

We translate the problem of calculating the entropy of a set of binary configurations/signals into a sequence of supervised classification tasks. Subsequently, one can use virtually any machine learning classification algorithm for…

SINDy is a method for learning system of differential equations from data by solving a sparse linear regression optimization problem [Brunton et al., 2016]. In this article, we propose an extension of the SINDy method that learns systems of…

Entropy is a quantity for counting physical degrees of freedom in a system. At a finite temperature, one can use thermal entropy to study thermodynamical properties. At zero temperature, entanglement entropy is expected to provide a…

Discrete entropy estimation is a classic information theory problem, wherein the average information content of a discrete random variable is estimated from samples alone. Naive approaches, such as the plugin method, fail to account for the…

A new index based on empirical copulas, termed the Copula Statistic (CoS), is introduced for assessing the strength of multivariate dependence and for testing statistical independence. New properties of the copulas are proved. They allow us…

This paper studies the notion of computational entropy. Using techniques from convex optimization, we investigate the following problems: (a) Can we derandomize the computational entropy? More precisely, for the computational entropy, what…

The empirical copula process, a fundamental tool for copula inference, is studied in the high dimensional regime where the dimension is allowed to grow to infinity exponentially in the sample size. Under natural, weak smoothness assumptions…