Related papers: Do readers use character information when programm…
One of the limitations of transformer networks is the sequence length due to the quadratic nature of the attention matrix. Classical self attention uses the entire sequence length, however, the actual attention being used is sparse. Humans…
In this paper, we report a method of intuitively transmitting symbolic information to untrained users via only their hands without using any visual or auditory cues. Our simple concept is presenting three-dimensional letter trajectories to…
Backward slicing has been used extensively in program understanding, debugging and scaling up of program analysis. For large programs, the size of the conventional backward slice is about 25% of the program size. This may be too large to be…
Embedding models are crucial for tasks in Information Retrieval (IR) and semantic similarity measurement, yet their handling of longer texts and associated positional biases remains underexplored. In this study, we investigate the impact of…
In this paper, we study the surprising impact that truncating text embeddings has on downstream performance. We consistently observe across 6 state-of-the-art text encoders and 26 downstream tasks, that randomly removing up to 50% of…
Large language models use high-dimensional latent spaces to encode and process textual information. Much work has investigated how the conceptual content of words translates into geometrical relationships between their vector…
In psycholinguistic modeling, surprisal from larger pre-trained language models has been shown to be a poorer predictor of naturalistic human reading times. However, it has been speculated that this may be due to data leakage that caused…
When searching a target in a natural scene, both the target's visual properties and similarity to the background influence whether (and how fast) humans are able to find it. However, thus far it has been unclear whether searchers adjust the…
In this work we find that many current redactions of PDF text are insecure due to non-redacted character positioning information. In particular, subpixel-sized horizontal shifts in redacted and non-redacted characters can be recovered and…
The word-based account of saccades drawn by a central gravity of the PVL is supported by two pillars of evidences. The first is the finding of the initial fixation location on a word resembled a normal distribution (Rayner, 1979). The other…
In recent years, the field of generative artificial intelligence, particularly in the domain of image generation, has exerted a profound influence on society. Despite the capability of AI to produce images of high quality, the augmentation…
Automatic detection of font size finds many applications in the area of intelligent OCRing and document image analysis, which has been traditionally practiced over uncompressed documents, although in real life the documents exist in…
Nowadays, learning increasingly involves the usage of search engines and web resources. The related interdisciplinary research field search as learning aims to understand how people learn on the web. Previous work has investigated several…
In the present work we analyzed the pupil size behavior of forty subjects while they read well defined sentences with different contextual predictability (i.e., regular sentences and proverbs). In general, pupil size increased when reading…
Searching for all occurrences of a pattern in a text is a fundamental problem in computer science with applications in many other fields, like natural language processing, information retrieval and computational biology. Sampled string…
Recent large language models have been trained on vast datasets, but also often on repeated data, either intentionally for the purpose of upweighting higher quality data, or unintentionally because data deduplication is not perfect and the…
Several studies have investigated the reasons behind the effectiveness of fine-tuning, usually through the lens of probing. However, these studies often neglect the role of the size of the dataset on which the model is fine-tuned. In this…
Recently, recurrent large language models (Recurrent LLMs) with linear computational complexity have re-emerged as efficient alternatives to self-attention-based LLMs (Self-Attention LLMs), which have quadratic complexity. However,…
Judging the readability of text has many important applications, for instance when performing text simplification or when sourcing reading material for language learners. In this paper, we present a 518 participant study which investigates…
This work investigates the ``small-vs-large gap'', where repeating on fewer samples can lead to compute saving during training compared to using a larger dataset. This is observed across algorithmic tasks, architectures and optimizers and…