English
Related papers

Related papers: Astronomy in the Cloud: Using MapReduce for Image …

200 papers

The Big Data management is a problem right now. The Big Data growth is very high. It is very difficult to manage due to various characteristics. This manuscript focuses on Big Data analytics in cloud environment using Hadoop. We have…

Distributed, Parallel, and Cluster Computing · Computer Science 2016-10-17 Mansaf Alam , Kashish Ara Shakil

Most of the popular Big Data analytics tools evolved to adapt their working environment to extract valuable information from a vast amount of unstructured data. The ability of data mining techniques to filter this helpful information from…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-09-23 Taha Tekdogan , Ali Cakmak

In the last two decades, the continuous increase of computational power has produced an overwhelming flow of data which has called for a paradigm shift in the computing architecture and large scale data processing mechanisms. MapReduce is a…

Databases · Computer Science 2013-02-14 Sherif Sakr , Anna Liu , Ayman G. Fayoumi

Cross-matching operation, which is to find corresponding data for the same celestial object or region from multiple catalogues,is indispensable to astronomical data analysis and research. Due to the large amount of astronomical catalogues…

Instrumentation and Methods for Astrophysics · Physics 2023-01-19 Yajie Zhang , Ce Yu , Chao Sun , Jian Xiao , Kun Li , Yifei Mu , Chenzhou Cui

Today, big data is generated from many sources and there is a huge demand for storing, managing, processing, and querying on big data. The MapReduce model and its counterpart open source implementation Hadoop, has proven itself as the de…

Distributed, Parallel, and Cluster Computing · Computer Science 2014-08-04 Saeed Shahrivari , Saeed Jalili

A common approach in the design of MapReduce algorithms is to minimize the number of rounds. Indeed, there are many examples in the literature of monolithic MapReduce algorithms, which are algorithms requiring just one or two rounds.…

Distributed, Parallel, and Cluster Computing · Computer Science 2015-01-22 Matteo Ceccarello , Francesco Silvestri

Multi-wavelength astronomical studies brings a wealth of science within reach. One way to achieve a cross-wavelength analysis is via `stacking', i.e. combining precise positional information from an image at one wavelength with data from…

Cosmology and Nongalactic Astrophysics · Physics 2017-09-14 Song Chen , Jonathan T. L. Zwart , Mario G. Santos

Current and upcoming radio-interferometers are expected to produce volumes of data of increasing size that need to be processed in order to generate the corresponding sky brightness distributions through imaging. This represents an…

Instrumentation and Methods for Astrophysics · Physics 2023-01-18 Claudio Gheller , Giuliano Taffoni , David Goz

Cloud Computing is emerging as a new computational paradigm shift. Hadoop-MapReduce has become a powerful Computation Model for processing large data on distributed commodity hardware clusters such as Clouds. In all Hadoop implementations,…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-07-04 B. Thirumala Rao , L. S. S. Reddy

We present an algorithm capable of detecting diffuse, dim sources of any size in an astronomical image. These sources often defeat traditional methods for source finding, which expand regions around points of high intensity. Extended…

Instrumentation and Methods for Astrophysics · Physics 2016-01-05 T. Butler-Yeoman , M. Frean , C. P. Hollitt , D. W. Hogg , M. Johnston-Hollitt

The number of mobile devices (e.g., smartphones, wearable technologies) is rapidly growing. In line with this trend, a massive amount of spatial data is being collected since these devices allow users to geo-tag user-generated content.…

Databases · Computer Science 2016-12-20 Afsin Akdogan

Cloud has been a computational and storage solution for many data centric organizations. The problem today those organizations are facing from the cloud is in data searching in an efficient manner. A framework is required to distribute the…

Distributed, Parallel, and Cluster Computing · Computer Science 2014-03-24 Gita Shah , Annappa , K. C. Shet

Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed,…

Computer Vision and Pattern Recognition · Computer Science 2019-09-05 Silas Ørting , Andrew Doyle , Arno van Hilten , Matthias Hirth , Oana Inel , Christopher R. Madan , Panagiotis Mavridis , Helen Spiers , Veronika Cheplygina

Scientific exploitation of the ever increasing volumes of astronomical data requires efficient and practical methods for data access, visualisation, and analysis. Hierarchical sky tessellation techniques enable a multi-resolution approach…

Instrumentation and Methods for Astrophysics · Physics 2015-06-17 P. Fernique , M. G. Allen , T. Boch , A. Oberto , F-X. Pineau , D. Durand , C. Bot , L. Cambresy , S. Derriere , F. Genova , F. Bonnarel

This paper presents a hybrid approach to spatial indexing of two dimensional data. It sheds new light on the age old problem by thinking of the traditional algorithms as working with images. Inspiration is drawn from an analogous situation…

Data Structures and Algorithms · Computer Science 2016-11-17 Lukasz A. Machowski , Tshilidzi Marwala

With the growing amount of astronomical data, there is an increasing need for automated data processing pipelines, which can extract scientific information from observation data without human interventions. A critical aspect of these…

Instrumentation and Methods for Astrophysics · Physics 2024-05-07 Peng Jia , Yu Song , Jiameng Lv , Runyu Ning

MapReduce is a programming system for distributed processing large-scale data in an efficient and fault tolerant manner on a private, public, or hybrid cloud. MapReduce is extensively used daily around the world as an efficient distributed…

Databases · Computer Science 2016-05-04 Philip Derbeko , Shlomi Dolev , Ehud Gudes , Shantanu Sharma

Cloud computing is a cost-effective way for start-up life sciences laboratories to store and manage their data. However, in many instances the data stored over the cloud could be redundant which makes cloud-based data management inefficient…

Distributed, Parallel, and Cluster Computing · Computer Science 2017-03-30 Kashish Ara Shakil , Ari Ora , Mansaf Alam , Shabih Shakeel

Nowadays most of the cloud applications process large amount of data to provide the desired results. Data volumes to be processed by cloud applications are growing much faster than computing power. This growth demands new strategies for…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-07-05 B. Thirumala Rao , N. V. Sridevi , V. Krishna Reddy , L. S. S. Reddy

The primary use of technical computing in the oil and gas industries is for seismic imaging of the earth's subsurface, driven by the business need for making well-informed drilling decisions during petroleum exploration and production.…

Distributed, Parallel, and Cluster Computing · Computer Science 2012-04-06 Masnida Emami , Ali Setayesh , Nasrin Jaberi