Formal concept analysis for evaluating intrinsic dimension of a natural language
Computation and Language
2023-11-21 v1 Artificial Intelligence
Algebraic Topology
Abstract
Some results of a computational experiment for determining the intrinsic dimension of linguistic varieties for the Bengali and Russian languages are presented. At the same time, both sets of words and sets of bigrams in these languages were considered separately. The method used to solve this problem was based on formal concept analysis algorithms. It was found that the intrinsic dimensions of these languages are significantly less than the dimensions used in popular neural network models in natural language processing.
Keywords
Cite
@article{arxiv.2311.10862,
title = {Formal concept analysis for evaluating intrinsic dimension of a natural language},
author = {Sergei O. Kuznetsov and Vasilii A. Gromov and Nikita S. Borodin and Andrei M. Divavin},
journal= {arXiv preprint arXiv:2311.10862},
year = {2023}
}
Comments
Preprint, 10th International Conference on Pattern Recognition and Machine Intelligence (PReMI 2023)