Related papers: PHYSICS: Benchmarking Foundation Models on Univers…
Large Language Models (LLMs) have achieved remarkable progress on advanced reasoning tasks such as mathematics and coding competitions. Meanwhile, physics, despite being both reasoning-intensive and essential to real-world understanding,…
We introduce a benchmark framework developed by and for the scientific community to evaluate, monitor and steer large language model development in fundamental physics. Building on philosophical concepts of scientific understanding and…
Large language models (LLMs) have demonstrated remarkable capabilities in solving complex reasoning tasks, particularly in mathematics. However, the domain of physics reasoning presents unique challenges that have received significantly…
The study guide (textbook) is part of a set of materials designed to support high-quality practical training in physics. It includes a collection of tasks for organizing both in-class and independent work. The guide serves as a foundation…
Acquiring the mathematical, conceptual, and problem-solving skills required in university-level physics courses is hard work, and the average student often lacks the knowledge and study skills they need to succeed in the introductory…
Large Language Models (LLMs) have shown impressive performance in domains such as mathematics and programming, yet their capabilities in physics remain underexplored and poorly understood. Physics poses unique challenges that demand not…
Large language models demonstrate remarkable capabilities across various domains, especially mathematics and logic reasoning. However, current evaluations overlook physics-based reasoning - a complex task requiring physics theorems and…
We present **Lean4PHYS**, a comprehensive reasoning framework for college-level physics problems in Lean4. **Lean4PHYS** includes *LeanPhysBench*, a college-level benchmark for formal physics reasoning in Lean4, which contains 200…
We introduce a benchmark to evaluate the capability of AI to solve problems in theoretical physics, focusing on high-energy theory and cosmology. The first iteration of our benchmark consists of 57 problems of varying difficulty, from…
While multimodal LLMs (MLLMs) demonstrate remarkable reasoning progress, their application in specialized scientific domains like physics reveals significant gaps in current evaluation benchmarks. Specifically, existing benchmarks often…
We present SeePhys, a large-scale multimodal benchmark for LLM reasoning grounded in physics questions ranging from middle school to PhD qualifying exams. The benchmark covers 7 fundamental domains spanning the physics discipline,…
The discipline of physics stands as a cornerstone of human intellect, driving the evolution of technology and deepening our understanding of the fundamental principles of the cosmos. Contemporary literature includes some works centered on…
Physics problem-solving is a challenging domain for AI models, requiring integration of conceptual understanding, mathematical reasoning, and interpretation of physical diagrams. Existing evaluations fail to capture the full breadth and…
Fundamental physics today is best defined operationally: it is the program of identifying the microscopic degrees of freedom, symmetries, and dynamical laws that (i) reproduce the Standard Model (SM) of particle physics, General Relativity…
A core level of basic information for physics is identified, based on an analysis of the characteristics of the parameters space, time, mass and charge. At this level, it is found that certain symmetries operate, which can be used to…
The ability to categorize problems based upon underlying principles, rather than contexts, is considered a hallmark of expertise in physics problem solving. With inspiration from a classic study by Chi, Feltovich, and Glaser, we compared…
Understanding issues involved in expertise in physics problem solving is important for helping students become good problem solvers. In part 1 of this article, we summarize the research on problem-solving relevant for physics education…
The current evaluation of mathematical skills in LLMs is limited, as existing benchmarks are either relatively small, primarily focus on elementary and high-school problems, or lack diversity in topics. Additionally, the inclusion of visual…
We discuss the categorization of 20 quantum mechanics problems by physics professors and undergraduate students from two honors-level quantum mechanics courses. Professors and students were asked to categorize the problems based upon…
Simulating physical systems is a core component of scientific computing, encompassing a wide range of physical domains and applications. Recently, there has been a surge in data-driven methods to complement traditional numerical simulations…