Related papers: A Gradient Descent Method for Optimization of Mode…

Transport is an important function in many network systems and understanding its behavior on biological, social, and technological networks is crucial for a wide range of applications. However, it is a property that is not well-understood…

It is increasingly common to encounter data from dynamic processes captured by static cross-sectional measurements over time, particularly in biomedical settings. Recent attempts to model individual trajectories from this data use optimal…

Bilevel optimization has been applied to a wide variety of machine learning models, and numerous stochastic bilevel optimization algorithms have been developed in recent years. However, most existing algorithms restrict their focus on the…

The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network's prediction, when confronted with a learning task. This iterative change can be naturally…

Solving statistical learning problems often involves nonconvex optimization. Despite the empirical success of nonconvex statistical optimization methods, their global dynamics, especially convergence to the desirable local minima, remain…

This paper applies a discrete adjoint gradient computation method for a multi-class traffic flow model on road networks. Vehicle classes are characterized by their specific velocity functions, which depend on the total traffic density,…

We derive a convex optimization problem on a steady-state nonequilibrium network of biochemical reactions, with the property that energy conservation and the second law of thermodynamics both hold at the problem solution. This suggests a…

In this paper, we investigate the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent's local objective function, which cannot be…

In the recent years, various gradient descent algorithms including the methods of gradient descent, gradient descent with momentum, adaptive gradient (AdaGrad), root-mean-square propagation (RMSProp) and adaptive moment estimation (Adam)…

Control of multihop Wireless networks in a distributed manner while providing end-to-end delay requirements for different flows, is a challenging problem. Using the notions of Draining Time and Discrete Review from the theory of fluid…

We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs. By leveraging a functional steepest descent idea, we derive a simple…

Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete…

The brain is immensely complex, with diverse components and dynamic interactions building upon one another to orchestrate a wide range of functions and behaviors. Understanding patterns of these complex interactions and how they are…

Transport and mixing processes in fluid flows can be studied directly from Lagrangian trajectory data, such as obtained from particle tracking experiments. Recent work in this context highlights the application of graph-based approaches,…

We rigorously derive the dense graph limit of a discrete model describing the formation of biological transportation networks. The discrete model, defined on undirected graphs with pressure-driven flows, incorporates a convex energy…

Optimal Transport is a foundational mathematical theory that connects optimization, partial differential equations, and probability. It offers a powerful framework for comparing probability distributions and has recently become an important…

In contrast to conventional artificial neural networks, which are structurally static, we present two approaches for evolving small networks into larger ones during training. The first method employs an auxiliary weight that directly…

Two aspects of neural networks that have been extensively studied in the recent literature are their function approximation properties and their training by gradient descent methods. The approximation problem seeks accurate approximations…

In this paper, we consider the unconstrained distributed optimization problem, in which the exchange of information in the network is captured by a directed graph topology, thus, nodes can only communicate with their neighbors.…

Emerging applications of machine learning in numerous areas involve continuous gathering of and learning from streams of data. Real-time incorporation of streaming data into the learned models is essential for improved inference in these…