English
Related papers

Related papers: Implementation and evaluation of data-compression …

200 papers

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression;…

Numerical Analysis · Mathematics 2025-05-15 Alyson Fox , Peter Lindstrom

The rapid growth of scientific data is surpassing advancements in computing, creating challenges in storage, transfer, and analysis, particularly at the exascale. While data reduction techniques such as lossless and lossy compression help…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-03-11 Jieyang Chen , Qian Gong , Yanliang Li , Xin Liang , Lipeng Wan , Qing Liu , Norbert Podhorszki , Scott Klasky

First-order optimization methods are crucial for solving large-scale data processing problems, particularly those involving convex non-smooth composite objectives. For such problems with convex non-smooth composite objectives, we introduce…

Optimization and Control · Mathematics 2025-10-06 Endrit Dosti , Sergiy A. Vorobyov , Themistoklis Charalambous

For low-dimensional data sets with a large amount of data points, standard kernel methods are usually not feasible for regression anymore. Besides simple linear models or involved heuristic deep learning models, grid-based discretizations…

Machine Learning · Computer Science 2019-03-01 Bastian Bohn , Michael Griebel , Jens Oettershagen

Today's scientific simulations, for example in the high-performance exascale sector, produce huge amounts of data. Due to limited I/O bandwidth and available storage space, there is the necessity to reduce scientific data of high…

Distributed, Parallel, and Cluster Computing · Computer Science 2024-07-25 N. Böing , J. Holke , C. Hergl , L. Spataro , G. Gassner , A. Basermann

Molecular dynamics simulations yield large amounts of trajectory data. For their durable storage and accessibility an efficient compression algorithm is paramount. State of the art domain-specific algorithms combine quantization, Huffman…

Distributed, Parallel, and Cluster Computing · Computer Science 2016-01-13 Jan Huwald , Stephan Richter , Peter Dittrich

Modern compression systems use linear transformations in their encoding and decoding processes, with transforms providing compact signal representations. While multiple data-dependent transforms for image/video coding can adapt to diverse…

Image and Video Processing · Electrical Eng. & Systems 2024-11-26 Alessandro Gnutti , Fabrizio Guerrini , Riccardo Leonardi , Antonio Ortega

Graphics Processing Units allow for running massively parallel applications offloading the CPU from computationally intensive resources, however GPUs have a limited amount of memory. In this paper a trie compression algorithm for massively…

Data Structures and Algorithms · Computer Science 2017-02-20 Xavier Bellekens , Amar Seeam , Christos Tachtatzis , Robert Atkinson

Gradient aggregation has long been identified as a major bottleneck in today's large-scale distributed machine learning training systems. One promising solution to mitigate such bottlenecks is gradient compression, directly reducing…

Machine Learning · Computer Science 2024-10-30 Wenchen Han , Shay Vargaftik , Michael Mitzenmacher , Brad Karp , Ran Ben Basat

Present day computational fluid dynamics simulations generate extremely large amounts of data, sometimes on the order of TB/s. Often, a significant fraction of this data is discarded because current storage systems are unable to keep pace.…

Computational Engineering, Finance, and Science · Computer Science 2021-03-03 Heather Pacella , Alec Dunton , Alireza Doostan , Gianluca Iaccarino

Error-bounded lossy compression is one of the most efficient solutions to reduce the volume of scientific data. For lossy compression, progressive decompression and random-access decompression are critical features that enable on-demand…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-09-03 Daoce Wang , Pascal Grosset , Jesus Pulido , Jiannan Tian , Tushar M. Athawale , Jinda Jia , Baixi Sun , Boyuan Zhang , Sian Jin , Kai Zhao , James Ahrens , Fengguang Song

GPUs are uniquely suited to accelerate (SQL) analytics workloads thanks to their massive compute parallelism and High Bandwidth Memory (HBM) -- when datasets fit in the GPU HBM, performance is unparalleled. Unfortunately, GPU HBMs remain…

Convolutional Gridding is a technique (algorithm) extensively used in Radio Interferometric Image Synthesis for fast inversion of functions sampled with irregular intervals on the Fourier plane. In this thesis, we propose some modifications…

Instrumentation and Methods for Astrophysics · Physics 2021-11-09 Daniel Muscat

Researchers are increasingly incorporating numeric high-order data, i.e., numeric tensors, within their practice. Just like the matrix/vector (MV) paradigm, the development of multi-purpose, but high-performance, sparse data structures and…

Mathematical Software · Computer Science 2018-02-09 Adam P. Harrison , Dileepan Joseph

Machine learning (ML) models are widely used in many important domains. For efficiently processing these computational- and memory-intensive applications, tensors of these over-parameterized models are compressed by leveraging sparsity,…

Hardware Architecture · Computer Science 2021-08-11 Shail Dave , Riyadh Baghdadi , Tony Nowatzki , Sasikanth Avancha , Aviral Shrivastava , Baoxin Li

Data compression is a well-studied (and well-solved) problem in the setup of long coding blocks. But important emerging applications need to compress data to memory words of small fixed widths. This new setup is the subject of this paper.…

Information Theory · Computer Science 2017-01-12 Ori Rottenstreich , Yuval Cassuto

Data compression is becoming critical for storing scientific data because many scientific applications need to store large amounts of data and post process this data for scientific discovery. Unlike image and video compression algorithms…

Machine Learning · Computer Science 2022-12-22 Tania Banerjee , Jong Choi , Jaemoon Lee , Qian Gong , Jieyang Chen , Scott Klasky , Anand Rangarajan , Sanjay Ranka

Data compression and decompression have become vital components of big-data applications to manage the exponential growth in the amount of data collected and stored. Furthermore, big-data applications have increasingly adopted GPUs due to…

The periodic Gaussian process (PGP) has been increasingly used to model periodic data due to its high accuracy. Yet, computing the likelihood of PGP has a high computational complexity of $\mathcal{O}\left(n^{3}\right)$ ($n$ is the data…

Methodology · Statistics 2023-02-10 Yongxiang Li , Yuting Pu , Changming Cheng , Qian Xiao

Sparse matrix-vector multiplication (SpMV) is a central building block for scientific software and graph applications. Recently, heterogeneous processors composed of different types of cores attracted much attention because of their…

Mathematical Software · Computer Science 2015-09-15 Weifeng Liu , Brian Vinter