Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel version of Celeste, suitable for learning catalogs from modern large-scale astronomical datasets. Our algorithmic innovations include a fast numerical optimization routine for Bayesian posterior inference and a statistically efficient scheme for decomposing astronomical optimization problems into subproblems. Our scalable implementation is written entirely in Julia, a new high-level dynamic programming language designed for scientific and numerical computing. We use Julia's high-level constructs for shared and distributed memory parallelism, and demonstrate effective load balancing and efficient scaling on up to 8192 Xeon cores on the NERSC Cori supercomputer.
@article{arxiv.1611.03404,
title = {Learning an Astronomical Catalog of the Visible Universe through Scalable Bayesian Inference},
author = {Jeffrey Regier and Kiran Pamnany and Ryan Giordano and Rollin Thomas and David Schlegel and Jon McAuliffe and Prabhat},
journal= {arXiv preprint arXiv:1611.03404},
year = {2016}
}