Related papers: Nonextensive information theoretical machine
Let $X$ be a random vector with distribution $P_{\theta}$ where $\theta$ is an unknown parameter. When estimating $\theta$ by some estimator $\varphi(X)$ under a loss function $L(\theta,\varphi)$, classical decision theory advocates that…
E.T. Jaynes, originator of the maximum entropy interpretation of statistical mechanics, emphasized that there is an inevitable trade-off between the conflicting requirements of robustness and accuracy for any inferencing algorithm. This is…
We define an entropy based on a chosen governing probability distribution. If a certain kind of measurements follow such a distribution it also gives us a suitable scale to study it with. This scale will appear as a link function that is…
The local structure entropy is a new method which is proposed to identify the influential nodes in the complex networks. In this paper a new form of the local structure entropy of the complex networks is proposed based on the Tsallis…
Maximum entropy principles in nonextensive statistical physics are revisited as an application of the Tsallis relative entropy defined for non-negative matrices in the framework of matrix analysis. In addtition, some matrix trace…
In this paper we consider the dynamic Tsallis entropy and employ it for four model systems: (i) the motion of Brownian oscillator, (ii) the motion of Brownian oscillator with noise, (iii) the fluctuation of particle density in hydrodynamics…
Physics-informed neural networks (PINNs) constitute a flexible approach to both finding solutions and identifying parameters of partial differential equations. Most works on the topic assume noiseless data, or data contaminated with weak…
The theoretical basis for a candidate variational principle for the information bottleneck (IB) method is formulated within the ambit of the generalized nonadditive statistics of Tsallis. Given a nonadditivity parameter $ q $, the role of…
We combine an axiomatics of R\'{e}nyi with the $q$--deformed version of Khinchin axioms to obtain a measure of information (i.e., entropy) which accounts both for systems with embedded self-similarity and non-extensivity. We show that the…
We present a tensor network model (TNM) for forecasting nonlinear and chaotic dynamics, bridging quantum many-body methods with classical complex systems. The TNM leverages hierarchical tensor contractions to encode non-Markovian temporal…
The nonextensive one-dimensional version of a hydrodynamical model for multiparticle production processes is proposed and discussed. It is based on nonextensive statistics assumed in the form proposed by Tsallis and characterized by a…
Data fields sampled on irregularly spaced points arise in many applications in the sciences and engineering. For regular grids, Convolutional Neural Networks (CNNs) have been successfully used to gaining benefits from weight sharing and…
This paper investigates the statistical properties of non-linear transformations (NLT) of random variables, in order to establish useful tools for estimation and information theory. Specifically, the paper focuses on linear regression…
Dimension is an inherent bottleneck to some modern learning tasks, where optimization methods suffer from the size of the data. In this paper, we study non-isotropic distributions of data and develop tools that aim at reducing these…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
In this work we use the extremization method of various information-theoretic measures (Fisher information, Shannon entropy, Tsallis entropy) for $d$-dimensional quantum systems, which complementary describe the spreading of the quantum…
The inverse Stefan problem, as a typical phase-change problem with moving boundaries, finds extensive applications in science and engineering. Recent years have seen the applications of physics-informed neural networks (PINNs) to solving…
We investigate the limiting cases of Tsallis statistics. The viewpoint adopted is not the standard information-theoretic one, where one derives the distribution from a given measure of information. Instead the mechanical approach recently…
Interactive statistical decision making (ISDM) features algorithm-dependent data generated through interaction. Existing information-theoretic lower bounds in ISDM largely target expected risk, while tail-sensitive objectives are less…
We provide a new information-theoretic generalization error bound that is exactly tight (i.e., matching even the constant) for the canonical quadratic Gaussian (location) problem. Most existing bounds are order-wise loose in this setting,…