Related papers: Big Data Challenges of FAST
The Internet of Things, crowdsourcing, social media, public authorities, and other sources generate bigger and bigger data sets. Big and open data offers many benefits for emergency management, but also pose new challenges. This chapter…
To assist with the commissioning (Jiang et al. 2019) of the Five-hundred-meter Aperture Spherical radio Telescope (FAST), we performed a pulsar search, with the primary goal of developing and testing the pulsar data acquisition and…
In this paper we consider some of the issues of working with big data and big spatial data and highlight the need for an open and critical framework. We focus on a set of challenges underlying the collection and analysis of big data. In…
In the last decade a new generation of telescopes and sensors has allowed the production of a very large amount of data and astronomy has become, a data-rich science; this transition is often labeled as: "data revolution" and "data…
Apache Spark is a Big Data framework for working on large distributed datasets. Although widely used in the industry, it remains rather limited in the academic community or often restricted to software engineers. The goal of this paper is…
We describe PSR J1926-0652, a pulsar recently discovered with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Using sensitive single-pulse detections from FAST and long-term timing observations from the Parkes 64-m radio…
The vast volume of marine wireless sampling data and its continuously explosive growth herald the coming of the era of marine wireless big data. Two challenges imposed by these data are how to fast, reliably, and sustainably deliver them in…
Perhaps one of the mostly hotly debated topics in recent years has been the question of "GIS and Big Data". Much of the discussion has been about the data: huge volumes of 2D and 3D spatial data and spatio-temporal data are now being…
Improving survey specifications are causing an exponential rise in pulsar candidate numbers and data volumes. We study the candidate filters used to mitigate these problems during the past fifty years. We find that some existing methods…
Science is becoming very data intensive1. Today's astronomy datasets with tens of millions of galaxies already present substantial challenges for data mining. In less than 10 years the catalogs are expected to grow to billions of objects,…
As the largest radio telescope in the world, the Square Kilometre Array (SKA) will lead the next generation of radio astronomy. The feats of engineering required to construct the telescope array will be matched only by the techniques…
The modern practice of Radio Astronomy is characterized by extremes of data volume and rates, principally because of the direct relationship between the signal to noise ratio that can be achieved and the need to Nyquist sample the RF…
We present timing solutions and analyses of 11 pulsars discovered by the Five-hundred-meter Aperture Spherical radio Telescope (FAST). These pulsars were discovered using an ultra-wide bandwidth receiver in drift-scan observations made…
Discovery of pulsars is one of the main goals for large radio telescopes. The Five-hundred-meter Aperture Spherical radio Telescope (FAST), that incorporates an L-band 19-beam receiver with a system temperature of about 20~K, is the most…
By assuming the inverse square law of solar wind plasma density as representative of other stars, it is shown that just outside a star the {\it outward} deflection of a passing radio signal at $\nu\approx 1$~GHz (which is capable of…
Astrophysics and cosmology are rich with data. The advent of wide-area digital cameras on large aperture telescopes has led to ever more ambitious surveys of the sky. Data volumes of entire surveys a decade ago can now be acquired in a…
The Einstein Telescope is a proposed third generation gravitational wave detector that will operate in the region of 1 Hz to a few kHz. As well as the inspiral of compact binaries composed of neutron stars or black holes, the lower…
One of the most significant problems of Big Data is to extract knowledge through the huge amount of data. The usefulness of the extracted information depends strongly on data quality. In addition to the importance, data quality has recently…
The Pan-STARRS Data Processing System is responsible for the steps needed to downloaded, archive, and process all images obtained by the Pan-STARRS telescopes, including real-time detection of transient sources such as supernovae and moving…
Fast Radio Bursts are millisecond bursts of radio radiation at frequencies of about 1 GHz, recently discovered in pulsar surveys. They have not yet been definitively identified with any other astronomical object or phenomenon. The bursts…