Related papers: Batch is back: CasJobs, serving multi-TB data on t…

The Sloan Digital Sky Survey is the largest redshift survey conducted to date, and the principal survey observations have all been conducted on the dedicated SDSS 2.5m and 0.5m telescopes at Apache Point Observatory. While the whole survey…

Astronomy is changing. Large projects, large collaborations, and large budgets are becoming the norm. The Sloan Digital Sky Survey (SDSS) is one example of this new astronomy, and in operating the original survey, we put in place and…

We present the first global-scale database of 4.3 billion P- and S-wave picks extracted from 1.3 PB continuous seismic data via a cloud-native workflow. Using cloud computing services on Amazon Web Services, we launched ~145,000…

The Sloan Digital Sky Survey (SDSS) will provide the data to support detailed investigations of the distribution of luminous and non- luminous matter in the Universe: a photometrically and astrometrically calibrated digital imaging survey…

Streaming algorithms are fundamental in the analysis of large and online datasets. A key component of many such analytic tasks is $q$-MAX, which finds the largest $q$ values in a number stream. Modern approaches attain a constant runtime by…

Scientists increasingly rely on Python tools to perform scalable distributed memory array operations using rich, NumPy-like expressions. However, many of these tools rely on dynamic schedulers optimized for abstract task graphs, which often…

Here we present CaosDB, a Research Data Management System (RDMS) designed to ensure seamless integration of inhomogeneous data sources and repositories of legacy data. Its primary purpose is the management of data from biomedical sciences,…

Software engineering (SWE) has recently emerged as a crucial testbed for next-generation LLM agents, demanding inherent capabilities in two critical dimensions: sustained iterative problem-solving (e.g., >50 interaction rounds) and…

Context: The efficient processing of Big Data is a challenging task for SQL and NoSQL Databases, where competent software architecture plays a vital role. The SQL Databases are designed for structuring data and supporting vertical…

SDAMS is the ensemble of database + software packages aimed to the archiving, quick-look analysis, off-line analysis, network accessibility and plotting of the SPOrt produced data. Many of the aspects related to data archiving, analysis and…

Cloud has been a computational and storage solution for many data centric organizations. The problem today those organizations are facing from the cloud is in data searching in an efficient manner. A framework is required to distribute the…

In the rapidly evolving landscape of modern data-driven technologies, software relies on large datasets and constant data center operations using various database systems to support computation-intensive tasks. As energy consumption in…

SciDB is a scalable, computational database management system that uses an array model for data storage. The array data model of SciDB makes it ideally suited for storing and managing large amounts of imaging data. SciDB is designed to…

We describe the current status of CATS, a publicly accessible database (web-server http://cats.sao.ru) allowing one to search in hundreds of catalogs of astronomical objects discovered all along the electromagnetic spectrum. Our emphasis is…

Due to the surge of spatio-temporal data volume, the popularity of location-based services and applications, and the importance of extracted knowledge from spatio-temporal data to solve a wide range of real-world problems, a plethora of…

The collection of time series data increases as more monitoring and automation are being deployed. These deployments range in scale from an Internet of things (IoT) device located in a household to enormous distributed Cyber-Physical…

We present the third edition of the Sloan Digital Sky Survey (SDSS) Quasar Catalog. The catalog consists of the 46,420 objects in the SDSS Third Data Release that have luminosities larger than M_i = -22 (in a cosmology with H_0 = 70…

Multi-wavelength astronomical studies require cross-identification of detections of the same celestial objects in multiple catalogs based on spherical coordinates and other properties. Because of the large data volumes and spherical…

NoSQL databases are widely used for massive data storage and real-time web applications. Yet important aspects of these data structures are not well understood. For example, NoSQL databases write most of their data to a collection of files…

Scientists are increasingly leveraging advances in instruments, automation, and collaborative tools to scale up their experiments and research goals, leading to new bursts of discovery. Various scientific disciplines, including…