English
Related papers

Related papers: Approximate Partition Selection for Big-Data Workl…

200 papers

Due to the falling costs of data acquisition and storage, researchers and industry analysts often want to find all instances of rare events in large datasets. For instance, scientists can cheaply capture thousands of hours of video, but are…

Databases · Computer Science 2022-01-05 Daniel Kang , Edward Gan , Peter Bailis , Tatsunori Hashimoto , Matei Zaharia

In high-dimensional generalized linear models, it is crucial to identify a sparse model that adequately accounts for response variation. Although the best subset section has been widely regarded as the Holy Grail of problems of this type,…

Machine Learning · Statistics 2023-08-02 Junxian Zhu , Jin Zhu , Borui Tang , Xuanyu Chen , Hongmei Lin , Xueqin Wang

Order picking and order packing entail retrieving items from storage and packaging them according to customer requests. These activities have always been the main concerns of the companies in reducing warehouse management costs. This paper…

Optimization and Control · Mathematics 2022-06-24 Sen Xue , Chuanhou Gao

The increasing digitalization in industry and society leads to a growing abundance of data available to be processed and exploited. However, the high volume of data requires considerable computational resources for applying machine learning…

Machine Learning · Computer Science 2024-03-19 Joel Luís Carbonera

In statistical physics and information theory, although the exponent of the partition function is often of our primary interest, there are cases where one needs more detailed information. In this paper, we present a general framework to…

Information Theory · Computer Science 2012-02-06 Ryuhei Mori , Toshiyuki Tanaka

Approximate Bayesian computation performs approximate inference for models where likelihood computations are expensive or impossible. Instead simulations from the model are performed for various parameter values and accepted if they are…

Computation · Statistics 2015-12-16 Dennis Prangle

Large-scale approximate nearest neighbor search (ANN) has been gaining attention along with the latest machine learning researches employing ANNs. If the data is too large to fit in memory, it is necessary to search for the most similar…

Machine Learning · Computer Science 2025-01-29 Taiga Ikeda , Daisuke Miyashita , Jun Deguchi

Compression techniques that support fast random access are a core component of any information system. Current state-of-the-art methods group documents into fixed-sized blocks and compress each block with a general-purpose adaptive…

Data Structures and Algorithms · Computer Science 2015-03-19 Christopher Hoobin , Simon J. Puglisi , Justin Zobel

Although distance measures are used in many machine learning algorithms, the literature on the context-independent selection and evaluation of distance measures is limited in the sense that prior knowledge is used. In cluster analysis,…

Machine Learning · Computer Science 2021-08-24 Michael C. Thrun

In the analysis of large/big data sets, aggregation (replacing values of a variable over a group by a single value) is a standard way of reducing the size (complexity) of the data. Data analysis programs provide different aggregation…

Machine Learning · Computer Science 2023-03-29 Vladimir Batagelj

This paper introduces a novel K-means clustering algorithm, an advancement on the conventional Big-means methodology. The proposed method efficiently integrates parallel processing, stochastic sampling, and competitive optimization to…

Machine Learning · Computer Science 2024-03-28 Rustam Mussabayev , Ravil Mussabayev

Distributed sparse learning with a cluster of multiple machines has attracted much attention in machine learning, especially for large-scale applications with high-dimensional data. One popular way to implement sparse learning is to use…

Machine Learning · Statistics 2018-10-29 Shen-Yi Zhao , Gong-Duo Zhang , Ming-Wei Li , Wu-Jun Li

Supervised learning under measurement constraints is a common challenge in statistical and machine learning. In many applications, despite extensive design points, acquiring responses for all points is often impractical due to resource…

Methodology · Statistics 2025-03-19 Lin Wang

The problem of automatically clustering data is an age old problem. People have created numerous algorithms to tackle this problem. The execution time of any of this algorithm grows with the number of input points and the number of cluster…

Machine Learning · Computer Science 2014-12-08 Aditya AV Sastry , Kalyan Netti

Many organizations routinely analyze large datasets using systems for distributed data-parallel processing and clusters of commodity resources. Yet, users need to configure adequate resources for their data processing jobs. This requires…

Distributed, Parallel, and Cluster Computing · Computer Science 2022-06-02 Lauritz Thamsen , Dominik Scheinert , Jonathan Will , Jonathan Bader , Odej Kao

This thesis presents two similarity-based approaches to sparse data problems. The first approach is to build soft, hierarchical clusters: soft, because each event belongs to each cluster with some probability; hierarchical, because cluster…

cmp-lg · Computer Science 2008-02-03 Lillian Lee

Clustering, or grouping, dataset elements based on similarity can be used not only to classify a dataset into a few categories, but also to approximate it by a relatively large number of representative elements. In the latter scenario,…

Machine Learning · Computer Science 2019-09-13 Tim Jaschek , Marko Bucyk , Jaspreet S. Oberoi

In database management, record linkage aims to identify multiple records that correspond to the same individual. This task can be treated as a clustering problem, in which a latent entity is associated with one or more noisy database…

Methodology · Statistics 2021-03-02 Brenda Betancourt , Juan Sosa , Abel Rodríguez

This paper gives a new algorithm for sampling tree-weighted partitions of a large class of planar graphs. Formally, the tree-weighted distribution on $k$-partitions of a graph weights $k$-partitions proportional to the product of the number…

Data Structures and Algorithms · Computer Science 2026-05-08 Sarah Cannon , Topher Pankow , Wesley Pegden , Jamie Tucker-Foltz

In the last decade, various works have used statistics on relations to improve both the theory and practice of conjunctive query execution. Starting with the AGM bound which took advantage of relation sizes, later works incorporated…

Databases · Computer Science 2025-01-13 Kyle Deeds , Timo Camillo Merkl