Related papers: Universally optimal crossover designs under subjec…

Scaling Bayesian optimization to high dimensions is challenging task as the global optimization of high-dimensional acquisition function can be expensive and often infeasible. Existing methods depend either on limited active variables or…

We show that dropout training is best understood as performing MAP estimation concurrently for a family of conditional models whose objectives are themselves lower bounded by the original dropout objective. This discovery allows us to pick…

A combinatorial optimization problem becomes very difficult in situations where the energy landscape is rugged, and the global minimum locates in a narrow region of the configuration space. When using the quantum approximate optimization…

Consider the following variant of the set cover problem. We are given a universe $U=\{1,...,n\}$ and a collection of subsets $\mathcal{C} = \{S_1,...,S_m\}$ where $S_i \subseteq U$. For every element $u \in U$ we need to find a set $\phi(u)…

Dropout is known as an effective way to reduce overfitting via preventing co-adaptations of units. In this paper, we theoretically prove that the co-adaptation problem still exists after using dropout due to the correlations among the…

We consider relative or subjective optimization problems where the goal function and feasible set are dependent of the current state of the system under consideration. In general, they are formulated as quasi-equilibrium problems, hence…

We generalize the Maximum Principle for free end point optimal control problems involving sweeping systems derived in [9] to cover the case where the end point is constrained to take values in a certain set. As in [9], an ingenious smooth…

Dropout is a regularization technique widely used in training artificial neural networks to mitigate overfitting. It consists of dynamically deactivating subsets of the network during training to promote more robust representations. Despite…

Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks. This paper proposes a theoretical explanation for this phenomenon: we show that, under a generative…

Optimal two-treatment, $p$ period crossover designs for binary responses are determined. The optimal designs are obtained by minimizing the variance of the treatment contrast estimator over all possible allocations of $n$ subjects to $2^p$…

Algorithmic approaches endow deep learning systems with implicit bias that helps them generalize even in over-parametrized settings. In this paper, we focus on understanding such a bias induced in learning through dropout, a popular…

Dropout has recently emerged as a powerful and simple method for training neural networks preventing co-adaptation by stochastically omitting neurons. Dropout is currently not grounded in explicit modelling assumptions which so far has…

We prove two universal approximation theorems for a range of dropout neural networks. These are feed-forward neural networks in which each edge is given a random $\{0,1\}$-valued filter, that have two modes of operation: in the first each…

Crossover designs randomly assign each unit to receive a sequence of treatments. By comparing outcomes within the same unit, these designs can effectively eliminate between-unit variation and facilitate the identification of both…

The paper suggests a new --- to the best of the author's knowledge --- characterization of decisions which are optimal in the multi-objective optimization problem with respect to a definite proper preference cone, a Euclidean cone with a…

Among the major difficulties that one may encounter when estimating parameters in a nonlinear regression model are the nonuniqueness of the estimator, its instability with respect to small perturbations of the observations and the presence…

Copula modelling has in the past decade become a standard tool in many areas of applied statistics. However, a largely neglected aspect concerns the design of related experiments. Particularly the issue of whether the estimation of copula…

Dropout as regularization has been used extensively to prevent overfitting for training neural networks. During training, units and their connections are randomly dropped, which could be considered as sampling many different submodels from…

We identify locally $D$-optimal crossover designs for generalized linear models. We use generalized estimating equations to estimate the model parameters along with their variances. To capture the dependency among the observations coming…

We consider the problem of constructing optimal designs for model discrimination between competing regression models. Various new properties of optimal designs with respect to the popular $T$-optimality criterion are derived, which in many…