Related papers: ETL for the integration of remote sensing data
The sharp and recent increase in the availability of data captured by different sensors combined with their considerably heterogeneous natures poses a serious challenge for the effective and efficient processing of remotely sensed data.…
Modern software engineering of electronic structure codes has seen a paradigm shift from monolithic workflows towards object-based modularity. Software objectivity allows for greater flexibility in the application of electronic structure…
Modern TEM instrumentation can probe a wide range of structural, optical, and chemical properties with unprecedented resolution. However, each of these properties must be recorded in independent datasets using different detector modes with…
The decline of the number of newly discovered mineral deposits and increase in demand for different minerals in recent years has led exploration geologists to look for more efficient and innovative methods for processing different data…
Recently, we have seen an increasing need for fresh data exploration, where data analysts seek to explore the main characteristics or detect anomalies of data being actively collected. In addition to the common challenges in classic data…
Transfer learning (TL), the next frontier in machine learning (ML), has gained much popularity in recent years, due to the various challenges faced in ML, like the requirement of vast amounts of training data, expensive and time-consuming…
As modern data pipelines continue to collect, produce, and store a variety of data formats, extracting and combining value from traditional and context-rich sources such as strings, text, video, audio, and logs becomes a manual process…
This paper aims to propose a significant way of remote access and real time monitoring of a particular geographic area by integrating wireless sensor clouds with existing Telecom infrastructure and applications built around them through a…
Plant scientists use Functional Structural Plant Models (FSPMs) to model plant systems within a limited space-time range. To allow FSPMs to abstract complex plant systems beyond a single model's limitation, an integration that compounds…
The data needed for machine learning (ML) model training, can reside in different separate sites often termed data silos. For data-intensive ML applications, data silos pose a major challenge: the integration and transformation of data…
In the last decade, integrated logistics has become an important challenge in the development of wireless communication, identification and sensing technology, due to the growing complexity of logistics processes and the increasing demand…
This paper proposes a framework called Watts for implementing, comparing, and recombining open-ended learning (OEL) algorithms. Motivated by modularity and algorithmic flexibility, Watts atomizes the components of OEL systems to promote the…
With the advantages that cloud computing offers in terms of platform as a service, software as a service, and infrastructure as a service, data engineers and data scientists are able to leverage cloud computing for their ETL/ELT (extract,…
A data warehouse efficiently prepares data for effective and fast data analysis and modelling using machine learning algorithms. This paper discusses existing solutions for the Data Extraction, Transformation, and Loading (ETL) process and…
Enterprises increasingly adopt multi cloud architectures to take advantage of diverse database engines, regional availability, and cost models. In these environments, ETL pipelines must process large, distributed datasets while minimizing…
This paper aims to propose a significant way of remote access and real time monitoring of a particular geographic area by integrating wireless sensor clouds with existing Telecom infrastructure and applications built around them through a…
The integration of large language models (LLMs) into electronic design automation (EDA) has significantly advanced the field, offering transformative benefits, particularly in register transfer level (RTL) code generation and understanding.…
An inverse modeling technique is introduced that combines elements of coupled logistic map models and wavelet analysis for the purpose of analyzing partial synchronization states in high-dimensional systems. Using Embedded Complex Logistic…
In-situ Electron Energy Loss Spectroscopy (EELS) is an instrumental technique that has traditionally been used to understand how the choice of materials processing has the ability to change local structure and composition. However, more…
The high availability and scalability of weakly-consistent systems attracts system designers. Yet, writing correct application code for this type of systems is difficult; even how to specify the intended behavior of such systems is still an…