Related papers: State Variation Mining: On Information Divergence …
Transfer learning is a key component of modern machine learning, enhancing the performance of target tasks by leveraging diverse data sources. Simultaneously, overparameterized models such as the minimum-$\ell_2$-norm interpolator (MNI) in…
Understanding output variance is critical in modeling nonlinear dynamic systems, as it reflects the system's sensitivity to input variations and feature interactions. This work presents a methodology for dynamically determining relevance…
When designing a message transmission system, from the point of view of making sure that the information transmitted is as fresh as possible, two rules of thumb seem reasonable: use small buffers and adopt a last-in-first-out policy. In…
The development and spread of information and communication technologies have increased and diversified information. However, the increase in the volume and the selection of information does not necessarily promote understanding. In…
Latent space model plays a crucial role in network analysis, and accurate estimation of latent variables is essential for downstream tasks such as link prediction. However, the large number of parameters to be estimated presents a…
Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different probability distributions. In this work, we give an information-theoretic analysis of the…
In diffusion-based molecular communication, information transport is governed by diffusion through a fluid medium. The achievable data rates for these channels are very low compared to the radio-based communication system, since diffusion…
We introduce two new classes of measures of information for statistical experiments which generalise and subsume $\phi$-divergences, integral probability metrics, $\mathfrak{N}$-distances (MMD), and $(f,\Gamma)$ divergences between two or…
High-dimensional dense embeddings have become central to modern Information Retrieval, but many dimensions are noisy or redundant. Recently proposed DIME (Dimension IMportance Estimation), provides query-dependent scores to identify…
Large Language Models (LLMs) store an extensive amount of factual knowledge obtained from vast collections of text. To effectively utilize these models for downstream tasks, it is crucial to have reliable methods for measuring their…
Trajectories in human aimed movements are inherently variable. Using the concept of positional variance profiles, such trajectories are shown to be decomposable into two phases: In a first phase, the variance of the limb position over many…
This paper focuses on the problem of finding a particular data recommendation strategy based on the user preferences and a system expected revenue. To this end, we formulate this problem as an optimization by designing the recommendation…
Quantifying the information content in a neural network model is essentially estimating the model's Kolmogorov complexity. Recent success of prequential coding on neural networks points to a promising path of deriving an efficient…
We consider a special form of state discrimination in which after the measurement we are given additional information that may help us identify the state. This task plays a central role in the analysis of quantum cryptographic protocols in…
Feature-based transfer is one of the most effective methodologies for transfer learning. Existing studies usually assume that the learned new feature representation is \emph{domain-invariant}, and thus train a transfer model $\mathcal{M}$…
The principles are elaborated which underlie the applications of general nonclassical states to communication and measurement systems. Relevant classical communication concepts are reviewed. Communication and measurement processes are…
To improve the performance of multi-agent reinforcement learning under the constraint of wireless resources, we propose a message importance metric and design an importance-aware scheduling policy to effectively exchange messages. The key…
Millimeter wave signals with multiple transmit and receive antennas are considered as enabling technology for enhanced mobile broadband services in 5G systems. While this combination is mainly associated with achieving high data rates, it…
Transfer learning is a valuable tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small number of training examples in the latter. Yet,…
By employing various empirical estimators for the Mutual Information (MI) measure, we calculate and compare the estimates and their confidence intervals for both normal and non-normal bivariate data samples. We find that certain nonlinear…