Related papers: Not All Lotteries Are Made Equal
The world of empirical machine learning (ML) strongly relies on benchmarks in order to determine the relative effectiveness of different algorithms and methods. This paper proposes the notion of "a benchmark lottery" that describes the…
Predicting human decision-making under risk and uncertainty is a long-standing challenge in cognitive science, economics, and AI. While prior research has focused on numerically described lotteries, real-world decisions often rely on…
We prove a conjecture of W.~Hackbusch in a bigger generality than in our previous article. Here we consider Tensor Train (TT) model with an arbitrary number of leaves and a corresponding "almost binary tree" for Hierarchical Tucker (HT)…
The matching hypothesis in social psychology claims that people are more likely to form a committed relationship with someone equally attractive. Previous works on stochastic models of human mate choice process indicate that patterns…
Given a public transportation network of stations and connections, we want to find a minimum subset of stations such that each connection runs through a selected station. Although this problem is NP-hard in general, real-world instances are…
Lotteries are a prevalent form of gambling between a seller and buyers. Designing a lottery requires a model of how buyers make decisions when confronted with uncertain outcomes. Cumulative prospect theory (CPT) is a descriptive model that…
Heavily overparameterized language models such as BERT, XLNet and T5 have achieved impressive success in many NLP tasks. However, their high model complexity requires enormous computation resources and extremely long training time for both…
Let $G$ be a graph, which represents a social network, and suppose each node $v$ has a threshold value $\tau(v)$. Consider an initial configuration, where each node is either positive or negative. In each discrete time step, a node $v$…
For simple classification tasks, we show that users can benefit from the advantages of using small, local, generative language models instead of large commercial models without a trade-off in performance or introducing extra labelling…
We show that, in a resource allocation problem, the ex ante aggregate utility of players with cumulative-prospect-theoretic preferences can be increased over deterministic allocations by implementing lotteries. We formulate an optimization…
Reinforcement learning (RL) yields substantial improvements in large language models (LLMs) downstream task performance and alignment with human values. Surprisingly, such large gains result from updating only a small subnetwork comprising…
Data selection plays a crucial role in data-driven decision-making, including in large language models (LLMs), and is typically task-dependent. Properties such as data quality and diversity have been extensively studied and are known to…
One of the main drawbacks of the practical use of neural networks is the long time required in the training process. Such a training process consists of an iterative change of parameters trying to minimize a loss function. These changes are…
We show how the success of deep learning could depend not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of…
Small-world and scale-free networks are known to be more easily synchronized than regular lattices, which is usually attributed to the smaller network distance between oscillators. Surprisingly, we find that networks with a homogeneous…
Large pre-trained transformers have been receiving explosive attention in the past few years, due to their wide adaptability for numerous downstream applications via fine-tuning, but their exponentially increasing parameter counts are…
The interaction between transitivity and sparsity, two common features in empirical networks, implies that there are local regions of large sparse networks that are dense. We call this the blessing of transitivity and it has consequences…
Ensembling neural networks is an effective way to increase accuracy, and can often match the performance of individual larger models. This observation poses a natural question: given the choice between a deep ensemble and a single neural…
Thinking Tokens (TT) have been proposed as an unsupervised method to facilitate reasoning in language models. However, despite their conceptual appeal, our findings show that TTs marginally improves performance and consistently…
A key generative principle within social and other complex networks is transitivity, where friends of friends are more likely friends. We propose a new model for highly dense complex networks based on transitivity, called the Iterated Local…