相关论文: Variational Equalities of Directed Information and…
In recent years, neural networks have achieved remarkable progress in various fields and have also drawn much attention in applying them on scientific problems. A line of methods involving neural networks for solving partial differential…
In this paper, we explore the two-point zeroth-order gradient estimator and identify the distribution of random perturbations that minimizes the estimator's asymptotic variance as the perturbation stepsize tends to zero. We formulate it as…
Information theoretic quantities play an important role in various settings in machine learning, including causality testing, structure inference in graphical models, time-series problems, feature selection as well as in providing privacy…
We deal with pointwise approximation of solutions of scalar stochastic differential equations in the presence of informational noise about underlying drift and diffusion coefficients. We define a randomized derivative-free version of…
The Information Bottleneck (IB) is a method of lossy compression of relevant information. Its rate-distortion (RD) curve describes the fundamental tradeoff between input compression and the preservation of relevant information embedded in…
Fundamental rate-distortion-perception (RDP) trade-offs arise in applications requiring maintained perceptual quality of reconstructed data, such as neural image compression. When compressed data is transmitted over public communication…
We consider stochastic programs conditional on some covariate information, where the only knowledge of the possible relationship between the uncertain parameters and the covariates is reduced to a finite data sample of their joint…
The nascent field of Rate-Distortion-Perception (RDP) theory is seeing a surge of research interest due to the application of machine learning techniques in the area of lossy compression. The information RDP function characterizes the…
In this paper, we consider distributed optimization problems where $n$ agents, each possessing a local cost function, collaboratively minimize the average of the local cost functions over a connected network. To solve the problem, we…
A sum of lognormal random variables (RVs) appears in many problems of science and engineering. For example, it is invloved in computing the distribution of recevied signal and interference powers for radio channels subject to lognormal…
In this work, we focus on variational Bayesian inference on the sparse Deep Neural Network (DNN) modeled under a class of spike-and-slab priors. Given a pre-specified sparse DNN structure, the corresponding variational posterior contraction…
The advantage of using a Discrete Variable Representation (DVR) is that the Hamiltonian of two interacting particles can be constructed in a very simple form. However the DVR Hamiltonian is approximate and, as a consequence, the results…
Weighted Poincar\'e-type and related inequalities provide upper bounds of the variance of functions. Their application in sensitivity analysis allows for quickly identifying the active inputs. Although the efficiency in prioritizing inputs…
This paper is concerned with the efficient evaluation of higher-order derivatives of functions $f$ that are composed of matrix operations. I.e., we want to compute the $D$-th derivative tensor $\nabla^D f(X) \in \mathbb R^{N^D}$, where…
We study a stochastic program where the probability distribution of the uncertain problem parameters is unknown and only indirectly observed via finitely many correlated samples generated by an unknown Markov chain with $d$ states. We…
Mutual information is widely used, in a descriptive way, to measure the stochastic dependence of categorical random variables. In order to address questions such as the reliability of the descriptive value, one must consider…
Rate-distortion (R-D) function, a key quantity in information theory, characterizes the fundamental limit of how much a data source can be compressed subject to a fidelity criterion, by any compression algorithm. As researchers push for…
Direct Preference Optimization (DPO) has emerged as a more computationally efficient alternative to Reinforcement Learning from Human Feedback (RLHF) with Proximal Policy Optimization (PPO), eliminating the need for reward models and online…
We present the first calculation of a fully-unintegrated parton distribution (beam function) at next-to-next-to-leading order (NNLO). We obtain the fully-differential beam function for quark-initiated processes by matching it onto standard…
We introduce a variational theory for processes adapted to the multi-dimensional Brownian motion filtration. The theory provides a differential structure which describes the infinitesimal evolution of Wiener functionals at very small…