English
Related papers

Related papers: Data Provenance and Management in Radio Astronomy:…

200 papers

An essential part of building a data-driven organization is the ability to handle and process continuous streams of data to discover actionable insights. The explosive growth of interconnected devices and the social Web has led to a large…

Distributed, Parallel, and Cluster Computing · Computer Science 2019-07-23 Haruna Isah , Farhana Zulkernine

The last decades have seen a surge of interests in distributed computing thanks to advances in clustered computing and big data technology. Existing distributed algorithms typically assume {\it all the data are already in one place}, and…

Machine Learning · Computer Science 2019-05-07 Donghui Yan , Yingjie Wang , Jin Wang , Guodong Wu , Honggang Wang

Handling sparse and unstructured geometric data, such as point clouds or event-based vision, is a pressing challenge in the field of machine vision. Recently, sequence models such as Transformers and state-space models entered the domain of…

Computer Vision and Pattern Recognition · Computer Science 2024-11-25 Mark Schöne , Yash Bhisikar , Karan Bania , Khaleelulla Khan Nazeer , Christian Mayr , Anand Subramoney , David Kappel

Data streams (streaming data) consist of transiently observed, evolving in time, multidimensional data sequences that challenge our computational and/or inferential capabilities. In this paper we propose user friendly approaches for robust…

Applications · Statistics 2015-01-20 Daniel Kosiorowski

Stream processing is a computing paradigm that supports real-time data processing for a wide variety of applications. At Meta, it's used across the company for various tasks such as deriving product insights, providing and improving user…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-12-09 Animesh Dangwal , Yufeng Jiang , Charlie Arnold , Jun Fan , Mohamed Bassem , Aish Rajagopal

Streaming anomaly detection refers to the problem of detecting anomalous data samples in streams of data. This problem poses challenges that classical and deep anomaly detection methods are not designed to cope with, such as conceptual…

Machine Learning · Computer Science 2022-10-12 Joseph Gallego-Mejia , Oscar Bustos-Brinez , Fabio Gonzalez

High throughput and low latency data processing is essential for systems requiring live decision making, control, and machine learning-optimized data reduction. We focus on two distinct use cases for in-flight streaming data processing for…

Instrumentation and Detectors · Physics 2023-02-14 Jack Hirschman , Andrei Kamalov , Razib Obaid , Finn H. O'Shea , Ryan N Coffee

Radio interferometry is an observational technique used to study astrophysical phenomena. Data gathered by an interferometer requires substantial processing before astronomers can extract the scientific information from it. Data processing…

Instrumentation and Methods for Astrophysics · Physics 2024-12-24 Brian M. Kirk , Urvashi Rau , Ramyaa Ramyaa

Product distribution matching (PDM) is proposed to generate target distributions over large alphabets by combining the output of several parallel distribution matchers (DMs) with smaller output alphabets. The parallel architecture of PDM…

Information Theory · Computer Science 2017-02-27 Georg Böcherer , Patrick Schulte , Fabian Steiner

Streaming computing enables the real-time processing of large volumes of data and offers significant advantages for various applications, including real-time recommendations, anomaly detection, and monitoring. The multi-way stream join…

Databases · Computer Science 2024-11-26 Jinlong Hu , Tingfeng Qiu

Advancements in scientific instrument sensors and connected devices provide unprecedented insight into ongoing experiments and present new opportunities for control, optimization, and steering. However, the diversity of sensors and…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-05-04 Jakob R. Elias , Ryan Chard , Maksim Levental , Zhengchun Liu , Ian Foster , Santanu Chaudhuri

Advancements in onboard computing mean remote sensing agents can employ state-of-the-art computer vision and machine learning at the edge. These capabilities can be leveraged to unlock new rare, transient, and pinpoint measurements of…

Robotics · Computer Science 2025-09-04 Itai Zilberstein , Alberto Candela , Steve Chien

Data provenance (the process of determining the origin and derivation of data outputs) has applications across multiple domains including explaining database query results and auditing scientific workflows. Despite decades of research,…

Databases · Computer Science 2026-05-15 Chrysanthi Kosyfaki , Ruiyuan Zhang , Nikos Mamoulis , Xiaofang Zhou

Stream applications are widely deployed on the cloud. While modern distributed streaming systems like Flink and Spark Streaming can schedule and execute them efficiently, streaming dataflows are often dynamically changing, which may cause…

Systems and Control · Electrical Eng. & Systems 2021-03-17 Pengqi Lu , Liang Yuan , Yunquan Zhang , Hang Cao , Kun Li

The last two decades witnessed tremendous advances in the Information and Communications Technologies. Beside improvements in computational power and storage capacity, communication networks carry nowadays an amount of data which was not…

Networking and Internet Architecture · Computer Science 2019-03-01 Martino Trevisan

The use of in-situ digital sensors for water quality monitoring is becoming increasingly common worldwide. While these sensors provide near real-time data for science, the data are prone to technical anomalies that can undermine the…

The persistence diagram, which describes the topological features of a dataset, is a key descriptor in Topological Data Analysis. The "Discrete Morse Sandwich" (DMS) method has been reported to be the most efficient algorithm for computing…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-10-27 Eve Le Guillou , Pierre Fortin , Julien Tierny

Diffusion models (DMs) are a class of generative models that allow sampling from a distribution learned over a training set. When applied to solving inverse problems, the reverse sampling steps are modified to approximately sample from a…

Image and Video Processing · Electrical Eng. & Systems 2025-05-27 Ismail Alkhouri , Shijun Liang , Cheng-Han Huang , Jimmy Dai , Qing Qu , Saiprasad Ravishankar , Rongrong Wang

The ubiquitous deployment of monitoring devices in urban flow monitoring systems induces a significant cost for maintenance and operation. A technique is required to reduce the number of deployed devices, while preventing the degeneration…

Computer Vision and Pattern Recognition · Computer Science 2020-02-07 Kun Ouyang , Yuxuan Liang , Ye Liu , Zekun Tong , Sijie Ruan , Yu Zheng , David S. Rosenblum

In this paper, we present a vision for a new generation of multimodal streaming systems that embed MLLMs as first-class operators, enabling real-time query processing across multiple modalities. Achieving this is non-trivial: while recent…