Related papers: Development of an Astrophysical Specific Language …
Astronomy is entering a new era as multiple, large area, digital sky surveys are in production. The resulting datasets are truly remarkable in their own right; however, a revolutionary step arises in the aggregation of complimentary…
The new NASA Astrophysics Data System (ADS) is designed with a serviceoriented architecture (SOA) that consists of multiple customized Apache Solr search engine instances plus a collection of microservices, containerized using Docker, and…
Today, data guides the decision-making process of most companies. Effectively analyzing and manipulating data at scale to extract and exploit relevant knowledge is a challenging task, due to data characteristics such as its size, the rate…
The rapid growth in scale and complexity of both computational and observational astrophysics over the past decade necessitates efficient and intuitive methods for examining and visualizing large datasets. Here, I present {\it AstroBlend},…
This paper reviews recent advances in big data optimization, providing the state-of-art of this emerging field. The main focus in this review are optimization techniques being applied in big data analysis environments. Integer linear…
AstroCloud is a cyber-Infrastructure for Astronomy Research initiated by Chinese Virtual Observatory (China-VO) under funding support from NDRC (National Development and Reform commission) and CAS (Chinese Academy of Sciences). The ultimate…
In astronomical and cosmological studies one often wishes to infer some properties of an infinite-dimensional field indexed within a finite-dimensional metric space given only a finite collection of noisy observational data. Bayesian…
Data challenges are emerging as powerful tools with which to answer fundamental astronomical questions. Time-domain astronomy lends itself to data challenges, particularly in the era of classification and anomaly detection. With improved…
The analysis of complex multiphysics astrophysical simulations presents a unique and rapidly growing set of challenges: reproducibility, parallelization, and vast increases in data size and complexity chief among them. In order to meet…
Shrinking transistors, which powered the advancement of computing in the past half century, has stalled due to power wall; now extreme heterogeneity is promised to be the next driving force to feed the needs of ever-increasingly diverse…
Radio astronomy observatories with high throughput back end instruments require real-time data processing. While computing hardware continues to advance rapidly, development of real-time processing pipelines remains difficult and…
For the successful development of the astrophysics and, accordingly, for obtaining more complete knowledge of the Universe, it is extremely important to combine and comprehensively analyze information of various types (e.g., about charged…
We present a review of data types and statistical methods often encountered in astronomy. The aim is to provide an introduction to statistical applications in astronomy for statisticians and computer scientists. We highlight the complex,…
It is well known that the best way to understand astronomical data is through machine learning, where a "black box" is set up, inside which a kind of artificial intelligence learns how to interpret the features in the data. We suggest that…
Scientific endeavors such as large astronomical surveys generate databases on the terabyte scale. These, usually multidimensional databases must be visualized and mined in order to find interesting objects or to extract meaningful and…
Probabilistic programming is a growing area that strives to make statistical analysis more accessible, by separating probabilistic modelling from probabilistic inference. In practice this decoupling is difficult. No single inference…
Data is a precious resource in today's society, and is generated at an unprecedented and constantly growing pace. The need to store, analyze, and make data promptly available to a multitude of users introduces formidable challenges in…
Astronomical data take on a multitude of forms -- catalogs, data cubes, images, and simulations. The availability of software for rendering high-quality three-dimensional graphics lends itself to the paradigm of exploring the incredible…
The advent of large data-set in cosmology has meant that in the past 10 or 20 years our knowledge and understanding of the Universe has changed not only quantitatively but also, and most importantly, qualitatively. Cosmologists rely on data…
As the Science Mission Directorate contemplates establishing an open code policy, we consider it timely to share our experiences as the developers of the open-source partial differential equation solver Dedalus. Dedalus is a flexible…