Related papers: Optimization via Information Geometry
Life depends as much on the flow of information as on the flow of energy. Here we review the many efforts to make this intuition precise. Starting with the building blocks of information theory, we explore examples where it has been…
We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or…
Information geometry provides a tool to systematically investigate parameter sensitivity of the state of a system. If a physical system is described by a linear combination of eigenstates of a complex (that is, non-Hermitian) Hamiltonian,…
In this work we: (1) review likelihood-based inference for parameter estimation and the construction of confidence regions; and, (2) explore the use of techniques from information geometry, including geodesic curves and Riemann scalar…
Machine learning has enabled significant benefits in diverse fields, but, with a few exceptions, has had limited impact on computer architecture. Recent work, however, has explored broader applicability for design, optimization, and…
We review basic notions in the field of information geometry such as Fisher metric on statistical manifold, $\alpha$-connection and corresponding curvature following Amari's work . We show application of information geometry to asymptotic…
Stochastic optimisation algorithms are the de facto standard for machine learning with large amounts of data. Handling only a subset of available data in each optimisation step dramatically reduces the per-iteration computational costs,…
Motion path planning is an intrinsically geometric problem which is central for design of robot systems. Since the early years of AI, robotics together with computer vision have been the areas of computer science that drove its development.…
Optimization algorithms appear in the core calculations of numerous Artificial Intelligence (AI) and Machine Learning methods, as well as Engineering and Business applications. Following recent works on the theoretical deficiencies of AI, a…
In this paper the approach to solving several combinatorial optimization problems using the local search and the genetic algorithm techniques is proposed. Initially this approach was developed in purpose to overcome some difficulties…
Genetic algorithms are modeled after the biological evolutionary processes that use natural selection to select the best species to survive. They are heuristics based and low cost to compute. Genetic algorithms use selection, crossover, and…
Information geometry promotes an investigation of the geometric structure of statistical manifolds, providing a series of elucidations in various areas of scientific knowledge. In the physical sciences, especially in quantum theory, this…
Consider a network design application where we wish to lay down a minimum-cost spanning tree in a given graph; however, we only have stochastic information about the edge costs. To learn the precise cost of any edge, we have to conduct a…
Information-Geometric Optimization (IGO) is a unified framework of stochastic algorithms for optimization problems. Given a family of probability distributions, IGO turns the original optimization problem into a new maximization problem on…
Recently, it has been recognized that phase transitions play an important role in the probabilistic analysis of combinatorial optimization problems. However, there are in fact many other relations that lead to close ties between computer…
In this survey, we describe the fundamental differential-geometric structures of information manifolds, state the fundamental theorem of information geometry, and illustrate some use cases of these information manifolds in information…
We present a new class of stochastic, geometrically-driven optimization algorithms on the orthogonal group $O(d)$ and naturally reductive homogeneous manifolds obtained from the action of the rotation group $SO(d)$. We theoretically and…
The introduction of a metric onto the space of parameters in models in Statistical Mechanics and beyond gives an alternative perspective on their phase structure. In such a geometrization, the scalar curvature, R, plays a central role. A…
This paper provides an overview of modern digital geometry and topology through mathematical principles, algorithms, and measurements. It also covers recent developments in the applications of digital geometry and topology including image…
In all but the most trivial optimization problems, the structure of the solutions exhibit complex interdependencies between the input parameters. Decades of research with stochastic search techniques has shown the benefit of explicitly…