Related papers: Data Reduction and Analysis Graphical Organizer

In the coming decade, astronomical surveys of the sky will generate tens of terabytes of images and detect hundreds of millions of sources every night. The study of these sources will involve computation challenges such as anomaly detection…

The range to which the Laser Interferometer Gravitational-Wave Observatory (LIGO) can observe astrophysical systems varies over time, limited by noise in the instruments and their environments. Identifying and removing the sources of noise…

As new large-scale astronomical surveys greatly increase the number of objects targeted and discoveries made, the requirement for efficient follow-up observations is crucial. Adaptive optics imaging, which compensates for the image-blurring…

The SINFONI data reduction pipeline, as part of the ESO-VLT Data Flow System, provides recipes for Paranal Science Operations, and for Data Flow Operations at Garching headquarters. At Paranal, it is used for the quick-look data evaluation.…

The ESPRESSO instrument, to be commissioned in the next months at the ESO VLT, is bound to became a landmark in the field of high-resolution optical spectroscopy, both for its ground-breaking science objectives (search for Earth-like…

Modern interferometers routinely provide radio-astronomical images down to subarcsecond resolution. However, interferometers filter out spatial scales larger than those sampled by the shortest baselines, which affects the measurement of…

This paper claims that a new field of empirical software engineering research and practice is emerging: data mining using/used-by optimizers for empirical studies or DUO. For example, data miners can generate models that are explored by…

Recent all-sky and large-area astronomical surveys and their catalogued data over the whole range of electromagnetic spectrum are reviewed, from Gamma-ray to radio, such as Fermi-GLAST and INTEGRAL in Gamma-ray, ROSAT, XMM and Chandra in…

As network traffic monitoring software for cybersecurity, malware detection, and other critical tasks becomes increasingly automated, the rate of alerts and supporting data gathered, as well as the complexity of the underlying model,…

We present a technique to automatically minimise the re-computation when a data processing program is iteratively changed, or added to, as is often the case in exploratory data analysis in radio astronomy. A typical example is flagging and…

Astronomy is undergoing through a methodological revolution triggered by an unprecedented wealth of complex and accurate data. The new panchromatic, synoptic sky surveys require advanced tools for discovering patterns and trends hidden…

Both Dimensionality Reduction (DR) and Graph Drawing (GD) aim to visualize abstract, non-linear structures, yet rely on different optimization paradigms. This contrast is evident in Multidimensional Scaling (MDS), which typically depends on…

Nowadays there is no field research which is not flooded with data. Among the sciences, Astrophysics has always been driven by the analysis of massive amounts of data. The development of new and more sophisticated observation facilities,…

Distributed data processing systems like MapReduce, Spark, and Flink are popular tools for analysis of large datasets with cluster resources. Yet, users often overprovision resources for their data processing jobs, while the resource usage…

Radio astronomical observations have very poor signal to noise ratios, unlike in other disciplines. On the other hand, it is possible to observe the object of interest for long time intervals as well as using a wider bandwidth.…

Many real-world datasets can be naturally represented as graphs, spanning a wide range of domains. However, the increasing complexity and size of graph datasets present significant challenges for analysis and computation. In response, graph…

We review the current state of data mining and machine learning in astronomy. 'Data Mining' can have a somewhat mixed connotation from the point of view of a researcher in this field. If used correctly, it can be a powerful approach,…

Large astronomical instruments using tens to hundreds of optical or infrared science detectors pose specific challenges for detector control, where, in addition to performance, other engineering aspects like scalability, power consumption,…

Data wrangling is a time-consuming and challenging task in a data science pipeline. While many tools have been proposed to automate or facilitate data wrangling, they often misinterpret user intent, especially in complex tasks. We propose…

Over the last two decades, significant advances have been made in the design and analysis of fixed-parameter algorithms for a wide variety of graph-theoretic problems. This has resulted in an algorithmic toolbox that is by now…