Related papers: Challenges for LSST scale data sets
In this paper we consider some of the issues of working with big data and big spatial data and highlight the need for an open and critical framework. We focus on a set of challenges underlying the collection and analysis of big data. In…
Among solar system objects, comets coming from the Oort Cloud are an elusive population, intrinsically rare and difficult to detect. Nonetheless, as the more pristine objects we can observe, they encapsulate critical cues on the formation…
For light curve generation, a pre-planned photometry survey is needed nowadays, where all of the exposure coordinates have to be given and don't change during the survey. This thesis shows it is not required and we can data-mine these light…
The Wide Field Survey Telescope (WFST) is a dedicated photometric surveying facility built jointly by the University of Science and Technology of China (USTC) and the Purple Mountain Observatory (PMO). Since many of its scientific…
The Trojan clouds contain a surprisingly diverse population of objects. It is only by understanding this diversity can we unravel their history. The more objects Lucy visits the better we can constrain the formation and evolution of the…
The Sloan Digital Sky Survey science database is approaching 2TB. While the vast majority of queries normally execute in seconds or minutes, this interactive execution time can be disproportionately increased by a small fraction of queries…
Recent and forthcoming advances in instrumentation, and giant new surveys, are creating astronomical data sets that are not amenable to the methods of analysis familiar to astronomers. Traditional methods are often inadequate not merely…
Modern photometric multiband digital surveys produce large amounts of data that, in order to be effectively exploited, need automatic tools capable to extract from photometric data an objective classification. We present here a new method…
Nowadays there is no field research which is not flooded with data. Among the sciences, Astrophysics has always been driven by the analysis of massive amounts of data. The development of new and more sophisticated observation facilities,…
As the 2dF Galaxy Redshift Survey and Sloan Digital Sky Survey move toward completion, it is time to ask what the next generation of survey of large-scale structure should be. I discuss some of the cosmological justifications for such…
Big data are data on a massive scale in terms of volume, intensity, and complexity that exceed the capacity of standard software tools. They present opportunities as well as challenges to statisticians. The role of computational…
Data Challenge 1 (DC1) is the first synthetic dataset produced by the Rubin Observatory Legacy Survey of Space and Time (LSST) Dark Energy Science Collaboration (DESC). DC1 is designed to develop and validate data reduction and analysis and…
Development of the Rubin Observatory Legacy Survey of Space and Time (LSST) includes a series of Data Challenges (DC) arranged by various LSST Scientific Collaborations (SC) that are taking place during the projects preoperational phase.…
The advent of modern technology, permitting the measurement of thousands of characteristics simultaneously, has given rise to floods of data characterized by many large or even huge datasets. This new paradigm presents extraordinary…
Within the evolving landscape of deep learning, the dilemma of data quantity and quality has been a long-standing problem. The recent advent of Large Language Models (LLMs) offers a data-centric solution to alleviate the limitations of…
The Sloan Digital Sky Survey (SDSS) is the largest and most ambitious optical CCD survey undertaken to date. It will ultimately map out one quarter of the sky with precision photometry in five bands, high-quality astrometry, and spectra of…
Science is becoming very data intensive1. Today's astronomy datasets with tens of millions of galaxies already present substantial challenges for data mining. In less than 10 years the catalogs are expected to grow to billions of objects,…
The Large Synoptic Survey Telescope (LSST) will be a large, wide-field ground-based system designed to obtain, starting in 2015, multiple images of the sky that is visible from Cerro Pachon in Northern Chile. About 90% of the observing time…
Neuromorphic engineering has a data problem. Despite the meteoric rise in the number of neuromorphic datasets published over the past ten years, the conclusion of a significant portion of neuromorphic research papers still states that there…
Modern radio telescopes generate large amounts of data, with the next generation Very Large Array (ngVLA) and the Square Kilometre Array (SKA) expected to feed up to 292 GB of visibilities per second to the science data processor (SDP).…