Related papers: Information Theory and Statistical Physics - Lectu…

In this position paper, I first describe a new perspective on machine learning (ML) by four basic problems (or levels), namely, "What to learn?", "How to learn?", "What to evaluate?", and "What to adjust?". The paper stresses more on the…

This is the second in a series of papers intended to provide a basic overview of some of the major ideas in particle physics. Part I [arXiv:0810.3328] was primarily an algebraic exposition of gauge theories. We developed the group theoretic…

This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…

Recently, it has been recognized that phase transitions play an important role in the probabilistic analysis of combinatorial optimization problems. However, there are in fact many other relations that lead to close ties between computer…

Influence theory is a foundational theory of physics that is not based on traditional empirically defined concepts, such as positions in space and time, mass, energy, or momentum. Instead, the aim is to derive these concepts, and their…

This paper is about Information Geometry, a relatively new subject within mathematical statistics that attempts to study the problem of inference by using tools from modern differential geometry. This paper provides an overview of some of…

These notes on string theory are based on a series of talks I gave during my graduate studies. As the talks, this introductory essay is intended for young students and non-string theory physicists.

Turbulence theory is usually concerned with the statistical moments of the velocity or its fluctuations. One could also analyze the implicit probability distributions. This is the purview of information theory. Here we use information…

Information theory is rapidly approaching its 70th birthday. What are promising future directions for research in information theory? Where will information theory be having the most impact in 10-20 years? What new and emerging areas are…

With the help of a general expression of the entropies in extensive and nonextensive systems, some important relations between thermodynamics and statistical mechanics are revealed through the views of thermodynamics and statistic physics.…

This article presents a summary graph to show the relationships between Information Retrieval (IR) and other related disciplines. The figure tells the key differences between them and the conditions under which one would transition into…

There is widespread emphasis on reform in the teaching of introductory statistics at the college level. Underpinning this reform is a consensus among educators and practitioners that traditional curricular materials and pedagogical…

Physical theories that depend on many parameters or are tested against data from many different experiments pose unique challenges to statistical inference. Many models in particle physics, astrophysics and cosmology fall into one or both…

This text is a slightly edited version of lecture notes for a course I gave at ETH, during the Summer term 2001, to undergraduate Mathematics and Physics students. It covers a few selected topics from perturbation theory at an introductory…

Educating the next generation of scientists in statistical methodology is an important task. Educating their instructors in statistical content knowledge and pedagogical knowledge is as important and provides an indirect impact of students'…

A didactical survey of the foundations of Algorithmic Information Theory. These notes are short on motivation, history and background but introduce some of the main techniques and concepts of the field. The "manuscript" has been evolving…

Education is a goal-oriented field. But if we want to treat education scientifically so we can accumulate, evaluate, and refine what we learn, then we must develop a theoretical framework that is strongly rooted in objective observations…

Critical transitions, or large changes in the state of a system after a small change in the system's external conditions or parameters, commonly occur in a wide variety of disciplines, from the biological and social sciences to physics.…

We give a survey of the foundations of statistical queries and their many applications to other areas. We introduce the model, give the main definitions, and we explore the fundamental theory statistical queries and how how it connects to…

Graph theory provides fundamental concepts for many fields of science like statistical physics, network analysis and theoretical computer science. Here we give a pedagogical introduction to graph theory, divided into three sections. In the…