Related papers: Efficient Automatic Differentiation of Implicit Fu…
We propose a generalization of separability in the context of global optimization. Our results apply to objective functions implemented as differentiable computer programs. They are presented in the context of a simple branch and bound…
The Implicit and Inverse Function Theorems are special cases of a general Implicit/Inverse Function Theorem which can be easily derived from either theorem. The theorems can thus be easily deduced from each other via the generalized…
In this work, multi-variable derivative-free optimization algorithms for unconstrained optimization problems are developed. A novel procedure for approximating the gradient of multi-variable objective functions based on non-commutative maps…
When a distributed algorithm must be executed by strategic agents with misaligned interests, a social leader needs to introduce an appropriate tax/subsidy mechanism to incentivize agents to faithfully implement the intended algorithm so…
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research. Some examples include…
Typical models of learning assume incremental estimation of continuously-varying decision variables like expected rewards. However, this class of models fails to capture more idiosyncratic, discrete heuristics and strategies that people and…
We present a simple functional programming language, called Dual PCF, that implements forward mode automatic differentiation using dual numbers in the framework of exact real number computation. The main new feature of this language is the…
Automatic differentiation (AD) aims to compute derivatives of user-defined functions, but in Turing-complete languages, this simple specification does not fully capture AD's behavior: AD sometimes disagrees with the true derivative of a…
Stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets. However, their statistical properties are not well understood, in theory. And in practice, avoiding numerical instability requires…
In inductive learning of a broad concept, an algorithm should be able to distinguish concept examples from exceptions and noisy data. An approach through recursively finding patterns in exceptions turns out to correspond to the problem of…
A new method is presented for obtaining indefinite integrals of common special functions. The approach is based on a Lagrangian formulation of the general homogeneous linear ordinary differential equation of second order. A general integral…
Evidential reasoning is now a leading topic in Artificial Intelligence. Evidence is represented by a variety of evidential functions. Evidential reasoning is carried out by certain kinds of fundamental operation on these functions. This…
Indirect inference requires simulating realisations of endogenous variables from the model under study. When the endogenous variables are discontinuous functions of the model parameters, the resulting indirect inference criterion function…
Setting regularization parameters for Lasso-type estimators is notoriously difficult, though crucial in practice. The most popular hyperparameter optimization approach is grid-search using held-out validation data. Grid-search however…
Functional equations satisfied by additive functions have a special interest not only in the theory of functional equations, but also in the theory of (commutative) algebra because the fundamental notions such as derivations and…
We introduce Scruff, a new framework for developing AI systems using probabilistic programming. Scruff enables a variety of representations to be included, such as code with stochastic choices, neural networks, differential equations, and…
Feature engineering, a crucial step of machine learning, aims to extract useful features from raw data to improve data quality. In recent years, great efforts have been devoted to Automated Feature Engineering (AutoFE) to replace expensive…
Multilevel optimization has gained renewed interest in machine learning due to its promise in applications such as hyperparameter tuning and continual learning. However, existing methods struggle with the inherent difficulty of efficiently…
Under general conditions, the equation $g(x,y) = 0$ implicitly defines $y$ locally as a function of $x$. In this article, we express divided differences of $y$ in terms of bivariate divided differences of $g$, generalizing a recent result…
This paper develops and implements a practical simulation-based method for estimating dynamic discrete choice models. The method, which can accommodate lagged dependent variables, serially correlated errors, unobserved variables, and many…