Related papers: Maximum Entropy Least Squares Solutions of Overdet…
We present an analytical-numerical method providing robust upper estimates for the topological entropy or, more generally, uniform volume growth exponents of differentiable mappings. By introducing varying metrics, we simplify the analysis…
This paper proposes an estimation framework to assess the performance of sorting over perturbed/noisy data. In particular, the recovering accuracy is measured in terms of Minimum Mean Square Error (MMSE) between the values of the sorting…
In this paper we present new algorithms and analysis for the linear inverse sensor placement and scheduling problems over multiple time instances with power and communications constraints. The proposed algorithms, which deal directly with…
Ill-posed inverse problems of the form y = X p where y is J-dimensional vector of a data, p is m-dimensional probability vector which cannot be measured directly and matrix X of observable variables is a known J,m matrix, J < m, are…
The paper addresses the model reduction problem for linear and nonlinear systems using the notion of least squares moment matching. For linear systems, the main idea is to approximate a transfer function by ensuring that the interpolation…
We consider machine learning techniques to develop low-latency approximate solutions to a class of inverse problems. More precisely, we use a probabilistic approach for the problem of recovering sparse stochastic signals that are members of…
In this paper, we prove a new identity for the least-square solution of an over-determined set of linear equation $Ax=b$, where $A$ is an $m\times n$ full-rank matrix, $b$ is a column-vector of dimension $m$, and $m$ (the number of…
In this paper, the maximum L$q$-likelihood estimator (ML$q$E), a new parameter estimator based on nonextensive entropy [Kibernetika 3 (1967) 30--35] is introduced. The properties of the ML$q$E are studied via asymptotic analysis and…
Estimating linear, mean-square continuous functionals is a pivotal challenge in statistics. In high-dimensional contexts, this estimation is often performed under the assumption of exact model sparsity, meaning that only a small number of…
We develop the method of Maximum Entropy (ME) as a technique to generate approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
Within a framework of utmost generality, we show that the entropy maximization procedure with linear constraints uniquely leads to the Shannon-Boltzmann-Gibbs entropy. Therefore, the use of this procedure with linear constraints should not…
Maximum entropy distributions with discrete support in $m$ dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions…
This document introduces a strategy to solve linear optimization problems. The strategy is based on the bounding condition each constraint produces on each one of the problem's dimension. The solution of a linear optimization problem is…
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth…
We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y = f( x^\top \beta^* ) + \epsilon$, where $f$ is nonlinear. To recover $\beta^*$, we propose an…
The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning. There is in general no explicit expression for the optimal MEE estimate…
The problem of least squares regression of a $d$-dimensional unknown parameter is considered. A stochastic gradient descent based algorithm with weighted iterate-averaging that uses a single pass over the data is studied and its convergence…
In this paper, the sparse sensor placement problem for least-squares estimation is considered, and the previous novel approach of the sparse sensor selection algorithm is extended. The maximization of the determinant of the matrix which…
We consider the problem of approximating optimal in the Minimum Mean Squared Error (MMSE) sense nonlinear filters in a discrete time setting, exploiting properties of stochastically convergent state process approximations. More…
Linear minimum mean square error (LMMSE) estimation is often ill-conditioned, suggesting that unconstrained minimization of the mean square error is an inadequate approach to filter design. To address this, we first develop a unifying…