Related papers: Unveiling Scaling Laws of Parameter Identifiabilit…

This paper examines the quantization methods used in large-scale data analysis models and their hyperparameter choices. The recent surge in data analysis scale has significantly increased computational resource requirements. To address…

Modelling, parameter identification, and simulation play an important role in systems biology. Usually, the goal is to determine parameter values that minimise the difference between experimental measurement values and model predictions in…

The dynamics based on information transfer is proposed as an underlying mechanism for the scale-invariant dynamic critical behavior observed in a variety of systems. We apply the dynamics to the globally-coupled Ising model, which is…

Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in…

Sharpness-aware Minimization (SAM) improves generalization in large-scale model training by linking loss landscape geometry to generalization. However, challenges such as mislabeled noisy data and privacy concerns have emerged as…

In this paper, we study an asymptotic approximation of the Fisher information for the estimation of a scalar parameter using quantized measurements. We show that, as the number of quantization intervals tends to infinity, the loss of Fisher…

Deep graph models (e.g., graph neural networks and graph transformers) have become important techniques for leveraging knowledge across various types of graphs. Yet, the neural scaling laws on graphs, i.e., how the performance of deep graph…

High-throughput characterization often requires estimating parameters and model dimension from experimental data of limited quantity and quality. Such data may result in an ill-posed inverse problem, where multiple sets of parameters and…

Quantum metrology shows that by exploiting nonclassical resources it is possible to overcome the fundamental limit of precision found for classical parameter-estimation protocols. The scaling of the quantum Fisher information -- which…

We introduce a model for neural scaling laws under sparse activations. In the model, test loss is often dominated by rare coordinates that are never observed in the training input. This mechanism induces a novel bottleneck absent from dense…

The Fisher information matrix (FIM) plays an important role in the analysis of parameter inference and system design problems. In a number of cases, however, the statistical data distribution and its associated information matrix are either…

Motivated by problems in data clustering, we establish general conditions under which families of nonparametric mixture models are identifiable, by introducing a novel framework involving clustering overfitted \emph{parametric} (i.e.…

Domain shift, the mismatch between training and testing data characteristics, causes significant degradation in the predictive performance in multi-source imaging scenarios. In medical imaging, the heterogeneity of population, scanners and…

In distributed-parameter inverse problems in computational mechanics, spatially varying fields are inferred from noisy, indirect, and heterogeneous observations. The relevant identifiability question concerns which spatial perturbation…

Mathematical models are crucial for optimizing and controlling chemical processes, yet they often face significant limitations in terms of computational time, algorithm complexity, and development costs. Hybrid models, which combine…

Accurately estimating traffic variables across unequipped portions of a network remains a significant challenge due to the limited coverage of sensor-equipped links, such as loop detectors and probe vehicles. A common approach is to apply…

Motivation: Gene regulatory interactions are of fundamental importance to various biological functions and processes. However, only a few previous computational studies have claimed success in revealing genome-wide regulatory landscapes…

The importance of the quantum Fisher information metric is testified by the number of applications that this has in very different fields, ranging from hypothesis testing to metrology, passing through thermodynamics. Still, from the rich…

To be fully useful for public health practice, models for epidemic response must be able to do more than predict -- it is also important to incorporate the mechanisms underlying transmission dynamics to enable policymakers and practitioners…

Identifying latent variables and causal structures from observational data is essential to many real-world applications involving biological data, medical data, and unstructured data such as images and languages. However, this task can be…