Related papers: Building a scalable global data processing pipelin…
Radio astronomy observatories with high throughput back end instruments require real-time data processing. While computing hardware continues to advance rapidly, development of real-time processing pipelines remains difficult and…
As astronomy enters the petascale data era, astronomers are faced with new challenges relating to storage, access and management of data. A shift from the traditional approach of combining data and analysis at the desktop to the use of…
AstroCloud is a cyber-Infrastructure for Astronomy Research initiated by Chinese Virtual Observatory (China-VO) under funding support from NDRC (National Development and Reform commission) and CAS (Chinese Academy of…
We explore how astronomers take observational data from telescopes, process them into usable scientific data products, curate them for later use, and reuse data for further inquiry. Astronomers have invested heavily in knowledge…
It is developed a very complex system (hardware/software) to detect luminosity variations connected with the discovery of new planets outside the Solar System. Traditional imaging approaches are very demanding in terms of computing time;…
Big data is gaining overwhelming attention since the last decade. Almost all the fields of science and technology have experienced a considerable impact from it. The cloud computing paradigm has been targeted for big data processing and…
Massive multiple-input multiple-output (MIMO) systems are considered as one of the leading technologies employed in the next generations of wireless communication networks (5G), which promise to provide higher spectral efficiency, lower…
Various aspects of cosmology require comprehensive all-sky mapping of the cosmic web to considerable depths. In order to probe the whole extragalactic sky beyond 100 Mpc, one must draw on multiwavelength datasets and state-of-the-art…
We have developed a method that maps large astronomical images onto a two-dimensional map and clusters them. A combination of various state-of-the-art machine learning (ML) algorithms is used to develop a fully unsupervised image quality…
Data-intensive applications often require exploratory analysis of large datasets. If analysis is performed on distributed resources, data locality can be crucial to high throughput and performance. We propose a "data diffusion" approach…
The statistical description of Giant Molecular Cloud (GMC) properties relies heavily on the performance of automatic identification algorithms, which are often seriously affected by the survey design. The algorithm we designed, SCIMES…
The cosmic web that characterizes the large-scale structure of the Universe can be quantified by a variety of methods. For example, large redshift surveys can be used in combination with point process algorithms to extract long curvilinear…
Beamlines at synchrotron light source facilities are powerful scientific instruments used to image samples and observe phenomena at high spatial and temporal resolutions. Typically, these facilities are equipped only with modest compute…
Astrophysics has become a domain extremely rich of scientific data. Data mining tools are needed for information extraction from such large datasets. This asks for an approach to data management emphasizing the efficiency and simplicity of…
Star surveys and model analyses show that many stars have absolute stable fluxes as good as 3% in 0.3-35{\mu}m wavebands and about 1% in the visible wavebands. The relative flux calibrations between stars are better than 0.2%. Some stars…
Large, high-resolution space-based imaging surveys produce a volume of data that is difficult to present to the public in a comprehensible way. While megapixel-sized images can still be printed out or downloaded via the World Wide Web, this…
The New Vacuum Solar Telescope (NVST) is a 1-m solar telescope that aims to observe the fine structures in both the photosphere and the chromosphere of the Sun. The observational data acquired simultaneously from one channel for the…
Collaborations in astronomy and astrophysics are faced with numerous cyber infrastructure challenges, such as large data sets, the need to combine heterogeneous data sets, and the challenge to effectively collaborate on those large,…
This paper describes the VIS Processing Function (VIS PF) of the Euclid ground segment pipeline, which processes and calibrates raw data from the VIS camera. We present the algorithms used in each processing element, along with a…
The science goals for ground-based large-area surveys, such as the Dark Energy Survey, Pan-STARRS, and the Large Synoptic Survey Telescope, require calibration of broadband photometry that is stable in time and uniform over the sky to…