Related papers: TherML: Thermodynamics of Machine Learning

Beyond neural scaling laws, little is known about the laws underlying large language models (LLMs). We introduce Neural Thermodynamic Laws (NTL) -- a new framework that offers fresh insights into LLM training dynamics. On the theoretical…

Over the last decade, the long-running endeavour to automate high-level processes in machine learning (ML) has risen to mainstream prominence, stimulated by advances in optimisation techniques and their impact on selecting ML…

An exponential growth in computing power, which has brought more sophisticated and higher resolution simulations of the climate system, and an exponential increase in observations since the first weather satellite was put in orbit, are…

With the advance of the powerful heterogeneous, parallel and distributed computing systems and ever increasing immense amount of data, machine learning has become an indispensable part of cutting-edge technology, scientific research and…

For many decades, experimental solid mechanics has played a crucial role in characterizing and understanding the mechanical properties of natural and novel materials. Recent advances in machine learning (ML) provide new opportunities for…

This review covers the new developments in machine learning (ML) that are impacting the multi-disciplinary area of aerospace engineering, including fundamental fluid dynamics (experimental and numerical), aerodynamics, acoustics, combustion…

Traditional machine learning mainly supervised learning, follows the assumptions of closed-world learning, i.e., for each testing class, a training class is available. However, such machine learning models fail to identify the classes which…

Machine learning (ML) in its current form implies that an answer to any problem can be well approximated by a function of a very peculiar form: a specially adjusted iteration of Heavyside theta-functions. It is natural to ask if the answers…

As machine learning technology gets applied to actual products and solutions, new challenges have emerged. Models unexpectedly fail to generalize to small changes in the distribution, tend to be confident on novel data they have never seen,…

As ultracold atom experiments become highly controlled and scalable quantum simulators, they require sophisticated control over high-dimensional parameter spaces and generate increasingly complex measurement data that need to be analyzed…

Some of the oldest and most important applications of thermodynamics are operations of refrigeration as well as production of useful energy. Part of the efforts to understand and develop thermodynamics in the quantum regime have been…

Machine learning is poised as a very powerful tool that can drastically improve our ability to carry out scientific research. However, many issues need to be addressed before this becomes a reality. This article focuses on one particular…

Weather is a phenomenon that affects everything and everyone around us on a daily basis. Weather prediction has been an important point of study for decades as researchers have tried to predict the weather and climatic changes using…

Over the past two decades, Machine Learning (ML) techniques have been increasingly utilized for the purpose of predicting outcomes in sport. In this paper, we provide a review of studies that have used ML for predicting results in team…

This document aims to be a self-contained, mathematically precise overview of transformer architectures and algorithms (*not* results). It covers what transformers are, how they are trained, what they are used for, their key architectural…

In this paper the analogy between a thermal engine and a waterwheel is developed in details, showing that the analogous of the flow of water in an hydraulic engine is the flow of entropy in a thermal one. This analogy mat serve to analyse…

This article develops iterative machine learning (IML) for output tracking. The input-output data generated during iterations to develop the model used in the iterative update. The main contribution of this article to propose the use of…

Multi-task learning (MTL) aims at learning related tasks in a unified model to achieve mutual improvement among tasks considering their shared knowledge. It is an important topic in recommendation due to the demand for multi-task prediction…

Metallic glasses are a promising class of materials celebrated for their exceptional thermal and mechanical properties. However, accurately predicting and understanding the melting temperature (T_m) and glass transition temperature (T_g)…

Machine learning (ML) has emerged as a pervasive tool in science, engineering, and beyond. Its success has also led to several synergies with molecular dynamics (MD) simulations, which we use to identify and characterize the major…