Related papers: DAMEWARE - Data Mining & Exploration Web Applicati…
Where appropriate repositories are not available to support all relevant astronomical data products, data can fall into darkness: unseen and unavailable for future reference and re-use. Some data in this category are legacy or old data, but…
This vision paper introduces a pioneering data lake architecture designed to meet Life \& Earth sciences' burgeoning data management needs. As the data landscape evolves, the imperative to navigate and maximize scientific opportunities has…
Here we describe the SHARE system, a web service based framework for distributed querying and reasoning on the semantic web. The main innovations of SHARE are: (1) the extension of a SPARQL query engine to perform on-demand data retrieval…
Software repository mining is the foundation for many empirical software engineering studies. The collection and analysis of detailed data can be challenging, especially if data shall be shared to enable replicable research and open science…
Smartphones have become the ultimate 'personal' computer, yet despite this, general-purpose data-mining and knowledge discovery tools for mobile devices are surprisingly rare. DataLearner is a new data-mining application designed…
We present recent results from the LCDM (Laboratory for Cosmological Data Mining; http://lcdm.astro.uiuc.edu) collaboration between UIUC Astronomy and NCSA to deploy supercomputing cluster resources and machine learning algorithms for the…
We describe the design and implementation of a high performance cloud that we have used to archive, analyze and mine large distributed data sets. By a cloud, we mean an infrastructure that provides resources and/or services over the…
This paper introduces Firmamento, a new online platform designed for astronomical research, particularly for studying blazars and other multi-messenger emitters. Firmamento provides access to a wealth of astronomical data, including…
In this article, we present a distributed framework for collecting and analyzing environmental and location data recorded by human users (carriers) with the use of portable sensors. We demonstrate the data mining analysis potential among…
Autonomous robot exploration requires a robot to efficiently explore and map unknown environments. Compared to conventional methods that can only optimize paths based on the current robot belief, learning-based methods show the potential to…
Due to the rapid growth of the World Wide Web, resource discovery becomes an increasing problem. As an answer to the demand for information management, a third generation of World-Wide Web tools will evolve: information gathering and…
Sky surveys represent a fundamental data basis for astronomy. We use them to map in a systematic way the universe and its constituents, and to discover new types of objects or phenomena. We review the subject, with an emphasis on the…
We are surrounded by overwhelming big data, which brings substantial advances but meanwhile poses many challenges. Geospatial big data comprises a big portion of big data, and is essential and powerful for decision-making if being utilized…
This is a companion Focus Demonstration article to the CANFAR+Skytree poster (Ball 2012), demonstrating the usage of the Skytree machine learning software on the Canadian Advanced Network for Astronomical Research (CANFAR) cloud computing…
This article presents DataXploreFines, an innovative Shiny application that revolutionizes data exploration, analysis, and visualization. The application offers functionalities for data loading, management, summarization, basic graphs,…
We present DisPerSE, a novel approach to the coherent multi-scale identification of all types of astrophysical structures, and in particular the filaments, in the large scale distribution of matter in the Universe. This method and…
Collaborations in astronomy and astrophysics are faced with numerous cyber infrastructure challenges, such as large data sets, the need to combine heterogeneous data sets, and the challenge to effectively collaborate on those large,…
Data warehouse architectural choices and optimization techniques are critical to decision support query performance. To facilitate these choices, the performance of the designed data warehouse must be assessed. This is usually done with the…
As modern astronomy confronts unprecedented data volumes, automated pipelines and machine-learning techniques have become essential for processing and analysis. As these workflows grow more complex, astronomers also require input and…
Modern machine learning algorithms are increasingly computationally demanding, requiring specialized hardware and distributed computation to achieve high performance in a reasonable time frame. Many hyperparameter search algorithms have…