English
Related papers

Related papers: FastqZip: An Improved Reference-Based Genome Seque…

200 papers

We propose a new compression scheme for genomic data given as sequence fragments called reads. The scheme uses a reference genome at the decoder side only, freeing the encoder from the burdens of storing references and performing…

Information Theory · Computer Science 2023-02-10 Yotam Gershon , Yuval Cassuto

At the present scenario of the internet, there exist many optimization techniques to improve the Web speed but almost expensive in terms of bandwidth. So after a long investigation on different techniques to compress the data without any…

Information Theory · Computer Science 2014-05-20 Hemant Kumar Saini , Satpal Singh Kushwaha , C. Rama Krishna

The fast growth of computational power and scales of modern super-computing systems have raised great challenges for the management of exascale scientific data. To maintain the usability of scientific data, error-bound lossy compression is…

Machine Learning · Computer Science 2023-11-08 Jinyang Liu , Sheng Di , Sian Jin , Kai Zhao , Xin Liang , Zizhong Chen , Franck Cappello

As HPC systems continue to grow to exascale, the amount of data that needs to be saved or transmitted is exploding. To this end, many previous works have studied using error-bounded lossy compressors to reduce the data size and improve the…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-05-16 Chengming Zhang , Sian Jin , Tong Geng , Jiannan Tian , Ang Li , Dingwen Tao

Thanks to the rapid proliferation of connected devices, sensor-generated time series constitute a large and growing portion of the world's data. Often, this data is collected from distributed, resource-constrained devices and centralized at…

Performance · Computer Science 2018-08-09 Davis Blalock , Samuel Madden , John Guttag

What learning algorithms can be run directly on compressively-sensed data? In this work, we consider the question of accurately and efficiently computing low-rank matrix or tensor factorizations given data compressed via random projections.…

Machine Learning · Computer Science 2019-05-28 Vatsal Sharan , Kai Sheng Tai , Peter Bailis , Gregory Valiant

Because of the vast volume of data being produced by today's scientific simulations, lossy compression allowing user-controlled information loss can significantly reduce the data size and the I/O burden. However, for large-scale cosmology…

Information Theory · Computer Science 2017-08-08 Dingewn Tao , Sheng Di , Zizhong Chen , Franck Cappello

With small-scale quantum processors transitioning from experimental physics labs to industrial products, these processors allow us to efficiently compute important algorithms in various fields. In this paper, we propose a quantum algorithm…

Quantum Physics · Physics 2020-05-22 Aritra Sarkar , Zaid Al-Ars , Carmen G. Almudever , Koen Bertels

Relative Lempel-Ziv (RLZ) parsing is a dictionary compression method in which a string $S$ is compressed relative to a second string $R$ (called the reference) by parsing $S$ into a sequence of substrings that occur in $R$. RLZ is…

Data Structures and Algorithms · Computer Science 2022-08-25 Philip Bille , Inge Li Gørtz , Simon J. Puglisi , Simon R. Tarnow

We present an end-to-end image compression system based on compressive sensing. The presented system integrates the conventional scheme of compressive sampling and reconstruction with quantization and entropy coding. The compression…

Computer Vision and Pattern Recognition · Computer Science 2020-01-22 Xin Yuan , Raziel Haimi-Cohen

We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory. After considering different solutions inspired by the hashing literature, we propose a method…

Computation and Language · Computer Science 2016-12-19 Armand Joulin , Edouard Grave , Piotr Bojanowski , Matthijs Douze , Hérve Jégou , Tomas Mikolov

Genome sequence analysis has enabled significant advancements in medical and scientific areas such as personalized medicine, outbreak tracing, and the understanding of evolution. Unfortunately, it is currently bottlenecked by the…

Compressed sensing is a signal processing method that acquires data directly in a compressed form. This allows one to make less measurements than what was considered necessary to record a signal, enabling faster or more precise measurement…

Statistical Mechanics · Physics 2012-08-20 Florent Krzakala , Marc Mézard , François Sausset , Yifan Sun , Lenka Zdeborová

Large-scale scientific simulations generate massive datasets, posing challenges for storage and I/O. Traditional lossy compression struggles to advance more in balancing compression ratio, data quality, and adaptability to diverse…

Distributed, Parallel, and Cluster Computing · Computer Science 2025-04-21 Wenqi Jia , Zhewen Hu , Youyuan Liu , Boyuan Zhang , Jinzhen Wang , Jinyang Liu , Wei Niu , Stavros Kalafatis , Junzhou Huang , Sian Jin , Daoce Wang , Jiannan Tian , Miao Yin

Today's large-scale scientific applications running on high-performance computing (HPC) systems generate vast data volumes. Thus, data compression is becoming a critical technique to mitigate the storage burden and data-movement cost.…

Distributed, Parallel, and Cluster Computing · Computer Science 2023-05-04 Boyuan Zhang , Jiannan Tian , Sheng Di , Xiaodong Yu , Yunhe Feng , Xin Liang , Dingwen Tao , Franck Cappello

Phylogenetic trees represent evolutionary relationships among sets of organisms. Popular phylogenetic reconstruction approaches typically yield hundreds to thousands of trees on a common leafset. Storing and sharing such large collection of…

Populations and Evolution · Quantitative Biology 2019-10-18 Balanand Jha , David Fernández-Baca , Akshay Deepak , Kumar Abhishek

A genome read data set can be quickly and efficiently remapped from one reference to another similar reference (e.g., between two reference versions or two similar species) using a variety of tools, e.g., the commonly-used CrossMap tool.…

Genomics · Quantitative Biology 2023-11-21 Jeremie S. Kim , Can Firtina , Meryem Banu Cavlak , Damla Senol Cali , Can Alkan , Onur Mutlu

Compressing images at extremely low bitrates (< 0.1 bpp) has always been a challenging task since the quality of reconstruction significantly reduces due to the strong imposed constraint on the number of bits allocated for the compressed…

Image and Video Processing · Electrical Eng. & Systems 2020-06-16 Suraj Kiran Raman , Aditya Ramesh , Vijayakrishna Naganoor , Shubham Dash , Giridharan Kumaravelu , Honglak Lee

In the realm of deep neural network deployment, low-bit quantization presents a promising avenue for enhancing computational efficiency. However, it often hinges on the availability of training data to mitigate quantization errors, a…

Computer Vision and Pattern Recognition · Computer Science 2024-09-18 Yuhang Li , Youngeun Kim , Donghyun Lee , Souvik Kundu , Priyadarshini Panda

Phishing has grown significantly in the past few years and is predicted to further increase in the future. The dynamics of phishing introduce challenges in implementing a robust phishing detection system and selecting features which can…

Cryptography and Security · Computer Science 2020-07-24 Rizka Purwanto , Arindam Pal , Alan Blair , Sanjay Jha