Related papers: An Hierarchical Approach to Big Data
General perception systems such as Perceivers can process arbitrary modalities in any combination and are able to handle up to a few hundred thousand inputs. They achieve this generality by using exclusively global attention operations.…
Creating mobile robots which are able to find and manipulate objects in large environments is an active topic of research. These robots not only need to be capable of searching for specific objects but also to estimate their poses often…
Hierarchical time-series forecasting (HTSF) is an important problem for many real-world business applications where the goal is to simultaneously forecast multiple time-series that are related to each other via a hierarchical relation.…
Astronomy is entering a new era as multiple, large area, digital sky surveys are in production. The resulting datasets are truly remarkable in their own right; however, a revolutionary step arises in the aggregation of complimentary…
Hierarchical clustering is a common algorithm in data analysis. It is unique among many clustering algorithms in that it draws dendrograms based on the distance of data under a certain metric, and group them. It is widely used in all areas…
Objective: The objective was to develop a cloud-based, federated system to serve as a single point of search, discovery and analysis for data generated under the NIH Helping to End Addiction Long-term (HEAL) Initiative. Materials and…
As several large single-dish radio surveys begin operation within the coming decade, a wealth of radio data will become available and provide a new window to the Universe. In order to fully exploit the potential of these data sets, it is…
Dynamic models have been successfully used in producing estimates of HIV epidemics at national level, due to their epidemiological nature and their ability to simultaneously estimate prevalence, incidence, and mortality rates. Recently, HIV…
Astronomical data generally consists of 2 or more high-resolution axes, e.g., X,Y position on the sky or wavelength and position-along-one-axis (long-slit spectrometer). Analyzing these multi-dimension observations requires combining 3D…
Scientific knowledge is growing rapidly, making it difficult to track progress and high-level conceptual links across broad disciplines. While tools like citation networks and search engines help retrieve related papers, they lack the…
Large digital sky surveys, over a broad range of wavelengths, both from the ground and from space observatories, are becoming a major source of astronomical data. Some examples include the Sloan Digital Sky Survey (SDSS) and the Digital…
Sky surveys represent a fundamental data basis for astronomy. We use them to map in a systematic way the universe and its constituents, and to discover new types of objects or phenomena. We review the subject, with an emphasis on the…
We introduce AXS (Astronomy eXtensions for Spark), a scalable open-source astronomical data analysis framework built on Apache Spark, a widely used industry-standard engine for big data processing. Building on capabilities present in Spark,…
We investigate the problem of multiplex graph embedding, that is, graphs in which nodes interact through multiple types of relations (dimensions). In recent years, several methods have been developed to address this problem. However, the…
Most domains of science are experiencing a paradigm shift due to the advent of a new generation of instruments and detectors which produce data and data streams at an unprecedented rate. The scientific exploitation of these data, namely…
This paper presents a hybrid approach to spatial indexing of two dimensional data. It sheds new light on the age old problem by thinking of the traditional algorithms as working with images. Inspiration is drawn from an analogous situation…
The next-generation astronomy digital archives will cover most of the universe at fine resolution in many wave-lengths, from X-rays to ultraviolet, optical, and infrared. The archives will be stored at diverse geographical locations. One of…
Over the past two decades, the field of high-dimensional statistics has experienced substantial progress, driven largely by technological advances that have dramatically reduced the cost and effort for data collection and storage across a…
Remote sensing is a higher technology to produce knowledge for data mining applications. In principle hyperspectral images (HSIs) is a remote sensing tool that provides precise classification of regions. The HSI contains more than a hundred…
We report the results of a study to compare the performance of two common database indexing methods, HTM and HEALPix, on Solaris and Windows database servers installed with PostgreSQL, and a Windows Server installed with MS SQL Server. The…