Related papers: BERT in Plutarch's Shadows

Recently, Natural Language Processing (NLP) has witnessed an impressive progress in many areas, due to the advent of novel, pretrained contextual representation models. In particular, Devlin et al. (2019) proposed a model, called BERT…

Scarcity of resources such as annotated text corpora for under-resourced languages like Albanian is a serious impediment in computational linguistics and natural language processing research. This paper presents AlbNER, a corpus of 900…

The increased digitization of world's textual heritage poses significant challenges for both computer science and literary studies. Overall, there is an urgent need of computational techniques able to adapt to the challenges of historical…

BERT-based architectures currently give state-of-the-art performance on many NLP tasks, but little is known about the exact mechanisms that contribute to its success. In the current work, we focus on the interpretation of self-attention,…

English language is in the spotlight of the Natural Language Processing (NLP) community with other languages, like Greek, lagging behind in terms of offered methods, tools and resources. Due to the increasing interest in NLP, in this paper…

Logical systems with classical negation and means for sentential or propositional self-reference involve, in some way, paradoxical statements such as the liar. However, the paradox disappears if one replaces classical by an appropriate…

Public debate on the alleged decline of language skills among younger generations often focuses on university students, the most highly educated segment of the population. Rather than addressing the ill posed question of linguistic decline,…

Notes of my lectures at the CIME (Levico Terme, june 2015). The lectures gave an overview of the L\"uroth problem, its history, the counter-examples found in the 70's, and the recent developments on stable rationality following the new…

In this paper, we explore the idea of analysing the historical bias of contextual language models based on BERT by measuring their adequacy with respect to Early Modern (EME) and Modern (ME) English. In our preliminary experiments, we…

We re-evaluate the great Leibniz-Newton calculus debate, exactly three hundred years after it culminated, in 1712. We reflect upon the concept of invention, and to what extent there were indeed two independent inventors of this new…

In this paper the claim that Zeno's paradoxes have been solved is contested. Although no one has ever touched Zeno without refuting him (Whitehead), it will be our aim to show that, whatever it was that was refuted, it was certainly not…

Large language models (LLMs) have been leveraged for several years now, obtaining state-of-the-art performance in recognizing entities from modern documents. For the last few months, the conversational agent ChatGPT has "prompted" a lot of…

Recently, pre-trained models have been the dominant paradigm in natural language processing. They achieved remarkable state-of-the-art performance across a wide range of related tasks, such as textual entailment, natural language inference,…

Pre-trained language models (LMs) encode rich information about linguistic structure but their knowledge about lexical polysemy remains unclear. We propose a novel experimental setup for analysing this knowledge in LMs specifically trained…

In the universe of Natural Language Processing, Transformer-based language models like BERT and (Chat)GPT have emerged as lexical superheroes with great power to solve open research problems. In this paper, we specifically focus on the…

We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent…

Johann Kepler (1571-1630) is sometimes considered as a precursor of science-fiction novels with the writing of "Somnium, sive opus posthumum of astronomia lunaris". In this work published posthumously in 1634 by his son Ludwig, Kepler…

The present article introduces ptarithmetic (short for "polynomial time arithmetic") -- a formal number theory similar to the well known Peano arithmetic, but based on the recently born computability logic (see…

This is a short metaphysical fable accentuating the conceptual difficulties involved, and succinctly criticizing the approaches pursued since the inception of quantum theory, in dealing with what is known collectively as `the measurement…

Recent progress in pretraining language models on large textual corpora led to a surge of improvements for downstream NLP tasks. Whilst learning linguistic knowledge, these models may also be storing relational knowledge present in the…