Related papers: Cross-Entropy method: convergence issues for exten…

Scalability is a major challenge in modern recommender systems. In sequential recommendations, full Cross-Entropy (CE) loss achieves state-of-the-art recommendation quality but consumes excessive GPU memory with large item catalogs,…

In this work we present a new method of black-box optimization and constraint satisfaction. Existing algorithms that have attempted to solve this problem are unable to consider multiple modes, and are not able to adapt to changes in…

Scalability issue plays a crucial role in productionizing modern recommender systems. Even lightweight architectures may suffer from high computational overload due to intermediate calculations, limiting their practicality in real-world…

We propose a method for the accurate estimation of rare event or failure probabilities for expensive-to-evaluate numerical models in high dimensions. The proposed approach combines ideas from large deviation theory and adaptive importance…

Model calibration aims to align confidence with prediction correctness. The Cross-Entropy (CE) loss is widely used for calibrator training, which enforces the model to increase confidence on the ground truth class. However, we find the CE…

This paper solves a new class of optimization problems under uncertainty, called Probable Event Constrained Optimization (PECO), which optimizes an objective function of decision variables and subjects to a set of Probable Event Constraints…

Cross-entropy method model predictive control (CEM--MPC) is a powerful gradient-free technique for nonlinear optimal control, but its performance is often limited by the reliance on random sampling. This conventional approach can lead to…

We construct a cross-entropy clustering (CEC) theory which finds the optimal number of clusters by automatically removing groups which carry no information. Moreover, our theory gives simple and efficient criterion to verify cluster…

Loss functions play a crucial role in deep metric learning thus a variety of them have been proposed. Some supervise the learning process by pairwise or tripletwise similarity constraints while others take advantage of structured similarity…

In this paper, we propose a novel method, aggregation cross-entropy (ACE), for sequence recognition from a brand new perspective. The ACE loss function exhibits competitive performance to CTC and the attention mechanism, with much quicker…

Variable selection is an important problem in statistics and machine learning. Copula Entropy (CE) is a mathematical concept for measuring statistical independence and has been applied to variable selection recently. In this paper we…

The probability of rare and extreme events is an important quantity for design purposes. However, computing the probability of rare events can be expensive because only a few events, if any, can be observed. To this end, it is necessary to…

In Part I (arXiv:1911.00619) of this article, we proposed an importance sampling algorithm to compute rare-event probabilities in forward uncertainty quantification problems. The algorithm, which we termed the "Bayesian Inverse Monte Carlo…

We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the…

Solving decision problems in complex, stochastic environments is often achieved by estimating the expected outcome of decisions via Monte Carlo sampling. However, sampling may overlook rare, but important events, which can severely impact…

Extreme weather events epitomize high cost: to society through their physical impacts, and to computer servers that simulate them to assess risk and advance physical understanding. It costs hundreds of simulation years to sample a few…

Quantifying synchronization phenomena based on the timing of events has recently attracted a great deal of interest in various disciplines such as neuroscience or climatology. A multitude of similarity measures has been proposed for this…

Probabilistic reasoning systems combine different probabilistic rules and probabilistic facts to arrive at the desired probability values of consequences. In this paper we describe the MESA-algorithm (Maximum Entropy by Simulated Annealing)…

Evaluating rare-event forecasts is challenging because standard metrics collapse as event prevalence declines. Measures such as F1-score, AUPRC, MCC, and accuracy induce degenerate thresholds -- converging to zero or one -- and their values…

We investigate a class of chance-constrained combinatorial optimization problems. Given a pre-specified risk level $\epsilon \in [0,1]$, the chance-constrained program aims to find the minimum cost selection of a vector of binary decisions…