Related papers: Piecewise linear approximations of the standard no…
Stochastic optimization often involves calculating the expected value of a first-order max or min function, known as a first-order loss function. In this context, loss functions are frequently approximated using piecewise linear functions.…
In this paper, we provide analytic expressions for the first-order loss function, the complementary loss function and the second-order loss function for several probability distributions. These loss functions are important functions in…
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules…
We introduce \textit{basic inequalities} for first-order iterative optimization algorithms, forming a simple and versatile framework that connects implicit and explicit regularization. While related inequalities appear in the literature, we…
This paper is devoted to studying the first-order variational analysis of non-convex and non-differentiable functions that may not be subdifferentially regular. To achieve this goal, we entirely rely on two concepts of directional…
This paper is devoted to second-order variational analysis of a rather broad class of extended-real-valued piecewise liner functions and their applications to various issues of optimization and stability. Based on our recent explicit…
Multi-task learning (MTL) has emerged as a pivotal paradigm in machine learning by leveraging shared structures across multiple related tasks. Despite its empirical success, the development of likelihood-based efficiently solvable…
It is shown how piecewise differentiable functions $F: \mathbb R^n \mapsto \mathbb R^m $ that are defined by evaluation programs can be approximated locally by a piecewise linear model based on a pair of sample points $\check x$ and $\hat…
In this work we study loss functions for learning and evaluating probability distributions over large discrete domains. Unlike classification or regression where a wide variety of loss functions are used, in the distribution learning and…
The analysis of complex nonlinear systems is often carried out using simpler piecewise linear representations of them. A principled and practical technique is proposed to linearize and evaluate arbitrary continuous nonlinear functions using…
For a variant of the algorithm in [Pit19] (arXiv:1903.10816) to compute the approximate density or distribution function of a linear mixture of independent random variables known by a finite sample, it is presented a proof of the functional…
We consider the integration of two-dimensional, piecewise constant functions with respect to copulas. By drawing a connection to linear assignment problems, we can give optimal upper and lower bounds for such integrals and construct the…
In this paper we consider stochastic composite convex optimization problems with the objective function satisfying a stochastic bounded gradient condition, with or without a quadratic functional growth property. These models include the…
Inspired by regularization techniques in statistics and machine learning, we study complementary composite minimization in the stochastic setting. This problem corresponds to the minimization of the sum of a (weakly) smooth function endowed…
The paper is devoted to a comprehensive second-order study of a remarkable class of convex extended-real-valued functions that is highly important in many aspects of nonlinear and variational analysis, specifically those related to…
A general piecewise (including pointwise) probability distribution with space-saving notation and its hierarchical particular cases are considered. The explicit closed-form normalization, expectation, and variance formulas along with the…
We design and mathematically analyze sampling-based algorithms for regularized loss minimization problems that are implementable in popular computational models for large data, in which the access to the data is restricted in some way. Our…
We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of $m$ samples containing both variables and $n$ samples missing one fixed…
Let $X$ be a random variable with distribution function $F,$ and $X_{1},X_{2},...,X_{n}$ are independent copies of $X.$ Consider the order statistics $X_{i:n},$ $i=1,2,...,n$ and denote $F_{i:n}(x)=P\{X_{i:n}\leq x\}.$ Using majorization…
Many common loss functions such as mean-squared-error, cross-entropy, and reconstruction loss are unnecessarily rigid. Under a probabilistic interpretation, these common losses correspond to distributions with fixed shapes and scales. We…