Related papers: Agencies and Science-Experiment Risk
It is not easy to rationalize how peer review, as the current grassroots of science, can work based on voluntary contributions of reviewers. There is no rationale to write impartial and thorough evaluations. Consequently, there is no risk…
Use of artificial intelligence is growing and expanding into applications that impact people's lives. People trust their technology without really understanding it or its limitations. There is the potential for harm and we are already…
As increasingly capable large language model (LLM)-based agents are developed, the potential harms caused by misalignment and loss of control grow correspondingly severe. To address these risks, we propose an approach that directly measures…
What should a court do with a preliminary-injunction request to halt a multi-billion-dollar particle-physics experiment that plaintiffs claim could create a black hole that will devour the planet? The real-life case of CERN's LHC seems like…
AI scientists powered by large language models have demonstrated substantial promise in autonomously conducting experiments and facilitating scientific discoveries across various disciplines. While their capabilities are promising, these…
Artificial intelligence is already being applied in and impacting many important sectors in society, including healthcare, finance, and policing. These applications will increase as AI capabilities continue to progress, which has the…
Recent work in AI safety has highlighted that in sequential decision making, objectives are often underspecified or incomplete. This gives discretion to the acting agent to realize the stated objective in ways that may result in undesirable…
Although AI systems are increasingly being leveraged to provide value to organizations, individuals, and society, significant attendant risks have been identified and have manifested. These risks have led to proposed regulations,…
Experiments in atomic physics have exceptional sensitivity to small shifts in energy in an atom, ion, or bound particle. They are particularly well suited to search for unique low-energy signatures of new physics, including effects that…
Neutrinos, being elusive subatomic particles having only weak interactions, provide an ideal platform to look for physics beyond the Standard Model. In the present era of neutrino physics, various experiments are focusing towards the…
The upcoming phase of space exploration not only includes trips to Mars and beyond, but also holds great promise for human progress. However, the harm caused by cosmic radiation, consisting of Galactic Cosmic Rays and Solar Particle Events,…
This position paper briefly discusses nudging, its use by autonomous agents, potential risks and ethical considerations while creating such systems. Instead of taking a normative approach, which guides all situations, the paper proposes a…
It is widely believed that theory is useful in physics because it describes simple systems and that strictly empirical phenomenological approaches are necessary for complex biological and social systems. Here we prove based upon an analysis…
This report examines what I see as the core argument for concern about existential risk from misaligned artificial intelligence. I proceed in two stages. First, I lay out a backdrop picture that informs such concern. On this picture,…
Judging the significance and reproducibility of quantitative research requires a good understanding of relevant uncertainties, but it is often unclear how well these have been evaluated and what they imply. Reported scientific uncertainties…
Academic research projects receive hundreds of billions of dollars of government investment each year. They complement business research projects by focusing on the generation of new foundational knowledge and addressing societal…
Many scientists have expressed concerns about potential catastrophic risks associated with new technologies. But expressing concern is one thing, identifying serious candidates another. Such risks are likely to be novel, rare, and difficult…
Are nuclear weapons useful for coercion, and, if so, what factors increase the credibility and effectiveness of nuclear threats? While prominent scholars like Thomas Schelling argue that nuclear brinkmanship, or the manipulation of nuclear…
Background: Natural or quasi experiments are appealing for public health research because they enable the evaluation of events or interventions that are difficult or impossible to manipulate experimentally, such as many policy and health…
Computational models are increasingly used in high-impact decision making in science, engineering, and medicine. The National Aeronautics and Space Administration (NASA) uses computational models to perform complex experiments that are…