Deep learning inference with the Event Horizon Telescope II. The Zingularity framework for Bayesian artificial neural networks

(abridged) In this second paper in our publication series, we present the open-source Zingularity framework for parameter inference with deep Bayesian artificial neural networks. We carried out out supervised learning with synthetic millimeter very long baseline interferometry observations of the EHT. Our ground-truth models are based on GRMHD simulations of Sgr A* and M87* on horizon scales. We investigated how well Zingularity neural networks are able to infer key model parameters from EHT observations, such as the black hole spin and the magnetic state of the accretion disk, when uncertainties in the data are accurately taken into account. Zingularity makes use of the TensorFlow Probability library and is able to handle large amounts of data with a combination of the efficient TFRecord data format plus the Horovod framework. Our approach is the first analysis of EHT data with Bayesian neural networks, where an unprecedented training data size, under consideration of a closely modeled EHT signal path, and the full information content of the observational data are used. Zingularity infers parameters based on salient features in the data and is containerized. Through parameter surveys and dedicated validation tests, we identified neural network architectures, that are robust against internal stochastic processes and unaffected by noise in the observational and model data. We give examples of how different data properties affect the network training. We show how the Bayesian nature of our networks gives trustworthy uncertainties and uncovers failure modes for uncharacterizable data. It is easy to achieve low validation errors during training on synthetic data with neural networks, particularly when the forward modeling is too simplified. Through careful studies, we demonstrate that our trained networks can generalize well so that reliable results can be obtained from observational data.