English
Related papers

Related papers: Seven Principles for Effective Scientific Big-Data…

200 papers

The optimization of composition and processing to obtain materials that exhibit desirable characteristics has historically relied on a combination of scientist intuition, trial and error, and luck. We propose a methodology that can…

Machine Learning · Statistics 2017-07-20 Julia Ling , Max Hutchinson , Erin Antono , Sean Paradiso , Bryce Meredig

Novel technologies in genomics allow creating data in exascale dimension with relatively minor effort of human and laboratory and thus monetary resources compared to capabilities only a decade ago. While the availability of this data…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-11-10 Sandra Gesing , Thomas Richard Connor , Ian Taylor

As observational datasets become larger and more complex, so too are the questions being asked of these data. Data simulations, i.e., synthetic data with properties (pixelization, noise, PSF, artifacts, etc.) akin to real data, are…

Instrumentation and Methods for Astrophysics · Physics 2019-10-25 Molly S. Peeples , Bjorn Emonts , Mark Kyprianou , Matthew T. Penny , Gregory F. Snyder , Christopher C. Stark , Michael Troxel , Neil T. Zimmerman , John ZuHone

The continuous increase of data generated provides enormous possibilities of both public and private companies. The management of this mass of data or big data will play a crucial role in the society of the future, as it finds applications…

Computers and Society · Computer Science 2015-01-15 Fatima El Jamiy , Abderrahmane Daif , Mohamed Azouazi , Abdelaziz Marzak

Recently, increasingly large amounts of data are generated from a variety of sources. Existing data processing technologies are not suitable to cope with the huge amounts of generated data. Yet, many research works focus on Big Data, a…

Distributed, Parallel, and Cluster Computing · Computer Science 2018-06-07 Wissem Inoubli , Sabeur Aridhi , Haithem Mezni , Mondher Maddouri , Engelbert Mephu Nguifo

The unparalleled success of artificial intelligence (AI) in the technology sector has catalyzed an enormous amount of research in the scientific community. It has proven to be a powerful tool, but as with any rapidly developing field, the…

Machine Learning · Computer Science 2022-10-07 M. R. Carbone

Society's capacity for algorithmic problem-solving has never been greater. Artificial Intelligence is now applied across more domains than ever, a consequence of powerful abstractions, abundant data, and accessible software. As capabilities…

Machine Learning · Statistics 2024-08-20 Kris Sankaran

Data volumes from multiple sky surveys have grown from gigabytes into terabytes during the past decade, and will grow from terabytes into tens (or hundreds) of petabytes in the next decade. This exponential growth of new data both enables…

Instrumentation and Methods for Astrophysics · Physics 2009-11-04 Kirk D. Borne

Data collected by large-scale instruments, observatories, and sensor networks are key enablers of scientific discoveries in many disciplines. However, ensuring that these data can be accessed, integrated, and analyzed in a democratized and…

Distributed, Parallel, and Cluster Computing · Computer Science 2021-12-14 Yubo Qin , Ivan Rodero , Manish Parashar

Experience suggests that structural issues in how institutional Astrophysics approaches data-driven science and the development of discovery technology may be hampering the community's ability to respond effectively to a rapidly changing…

Big medical data poses great challenges to life scientists, clinicians, computer scientists, and engineers. In this paper, a group of life scientists, clinicians, computer scientists and engineers sit together to discuss several fundamental…

While AI coding tools have demonstrated potential to accelerate software development, their use in scientific computing raises critical questions about code quality and scientific validity. In this paper, we provide ten practical rules for…

Software Engineering · Computer Science 2025-11-03 Eric W. Bridgeford , Iain Campbell , Zijao Chen , Zhicheng Lin , Harrison Ritz , Joachim Vandekerckhove , Russell A. Poldrack

As large-scale social data explode and machine-learning methods evolve, scholars of entrepreneurship and innovation face new research opportunities but also unique challenges. This chapter discusses the difficulties of leveraging…

General Economics · Economics 2025-05-14 Ningzi Li , Shiyang Lai , James Evans

The 2020s will be the most data-rich decade of astronomy in history. As the scale and complexity of our surveys increase, the problem of scheduling becomes more critical. We must develop high-quality scheduling approaches, implement them as…

Instrumentation and Methods for Astrophysics · Physics 2019-07-19 Eric C. Bellm , Eric B. Ford , Aaron Tohuvavohu , Michael W. Coughlin , Brett Morris , Bryan Miller , Jennifer Sobeck , Reed Riddle , Chuanfei Dong , Peter Yoachim

Scientific discovery is a complex cognitive process that has driven human knowledge and technological progress for centuries. While artificial intelligence (AI) has made significant advances in automating aspects of scientific reasoning,…

Machine Learning · Computer Science 2024-12-24 Chandan K Reddy , Parshin Shojaee

In this work, we identify elements of effective machine learning datasets in astronomy and present suggestions for their design and creation. Machine learning has become an increasingly important tool for analyzing and understanding the…

Instrumentation and Methods for Astrophysics · Physics 2022-11-30 Bernie Boscoe , Tuan Do , Evan Jones , Yunqi Li , Kevin Alfaro , Christy Ma

Data science has employed great research efforts in developing advanced analytics, improving data models and cultivating new algorithms. However, not many authors have come across the organizational and socio-technical challenges that arise…

Machine Learning · Computer Science 2022-01-17 Iñigo Martinez , Elisabeth Viles , Igor G. Olaizola

This chapter addresses the forth paradigm of materials research -- big-data driven materials science. Its concepts and state-of-the-art are described, and its challenges and chances are discussed. For furthering the field, Open Data and an…

Materials Science · Physics 2019-04-12 Claudia Draxl , Matthias Scheffler

The research in the sixth generation of communication networks needs to tackle new challenges in order to meet the requirements of emerging applications in terms of high data rate, low latency, high reliability, and massive connectivity. To…

Information Theory · Computer Science 2023-02-09 Marwa Chafii , Lina Bariah , Sami Muhaidat , Merouane Debbah

Today, data guides the decision-making process of most companies. Effectively analyzing and manipulating data at scale to extract and exploit relevant knowledge is a challenging task, due to data characteristics such as its size, the rate…

Software Engineering · Computer Science 2025-03-24 Arianna Dragoni , Alessandro Margara