Related papers: On Information Processing Limitations In Humans an…
Humans' distinctive role in the world can largely be attributed to our capacity for iterated learning, a process by which knowledge is expanded and refined over generations. A range of theories seek to explain why humans are so adept at…
In this Chapter, we ask questions (1) What is the right way to measure the quality of information processing in a biological system? and (2) What can real-life organisms do in order to improve their performance in information-processing…
Algorithmic information theory roots the concept of information in computation rather than probability. These lecture notes were constructed in conjunction with the graduate course I taught at Universit\`a della Svizzera italiana in the…
In computer science, we can theoretically neatly separate transmission and processing of information, hardware and software, and programs and their inputs. This is much more intricate in biology, Nevertheless, I argue that Shannon's concept…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
The objective of this paper is to explore the opportunities for human information behaviour research to inform and influence the field of machine learning and the resulting machine information behaviour. Using the development of foundation…
Artificial Intelligence began as a field probing some of the most fundamental questions of science - the nature of intelligence and the design of intelligent artifacts. But it has grown into a discipline that is deeply entwined with…
Traditional machine learning relies on explicit models and domain assumptions, limiting flexibility and interpretability. We introduce a model-free framework using surprisal (information theoretic uncertainty) to directly analyze and…
Many biological phenomena or social events critically depend on how information evolves in complex networks. However, a general theory to characterize information evolution is yet absent. Consequently, numerous unknowns remain about the…
Today, the vast majority of the world's digital information is represented using the fundamental assumption, introduced by Claude Shannon in 1948, that ``...the semantic aspects of communication are irrelevant to the engineering problem (of…
This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…
The human brain is the substrate for human intelligence. By simulating the human brain, artificial intelligence builds computational models that have learning capabilities and perform intelligent tasks approaching the human level. Deep…
Observations on the past provide some hints about what will happen in the future, and this can be quantified using information theory. The ``predictive information'' defined in this way has connections to measures of complexity that have…
The overarching problem in artificial intelligence (AI) is that we do not understand the intelligence process well enough to enable the development of adequate computational models. Much work has been done in AI over the years at lower…
Since Claude Shannon founded Information Theory, information theory has widely fostered other scientific fields, such as statistics, artificial intelligence, biology, behavioral science, neuroscience, economics, and finance. Unfortunately,…
Human consciousness has been a long-lasting mystery for centuries, while machine intelligence and consciousness is an arduous pursuit. Researchers have developed diverse theories for interpreting the consciousness phenomenon in human brains…
How much information is transmitted when animals use emotions to communicate? It is clear that emotions are used as communication systems in humans and other species. The quantitative theory of emotion information presented here is based on…
In this article we are investigating the computers development process in the past decades in order to identify the factors that influence it the most. We describe such factors and use them to predict the direction of further development.…
We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset.…
The thesis explores the role machine learning methods play in creating intuitive computational models of neural processing. Combined with interpretability techniques, machine learning could replace human modeler and shift the focus of human…