Quantifying the efficiency of written language
Datos Bibliográficos
| ID | 21226599 |
|---|---|
| Autores | Alexander Koplenig (0000-0002-9630-9680, Leibniz Institute for the German Language, autor de correspondencia) |
| Año | 2021 |
| Volumen | 7 |
| Número | s3 |
| Fecha de publicación | 2021-04-21 |
| Peer Reviewed | Sí |
| Open Access | No |
| Tipo | ARTICLE |
| Revista | Linguistics Vanguard (JOURNAL) |
| Identificadores de la revista | ISSN: 2199-174X • E-ISSN: 2199-174X |
| Editorial | Walter de Gruyter GmbH (PUBLISHER • DE) |
| DOI | 10.1515/lingvan-2019-0057 |
| OpenAlex | W3159346524 |
| Idioma | EN |
| Referencias citadas | 39 |
Information theory can be used to assess how efficiently a message is transmitted on the basis of different symbolic systems. In this paper, I estimate the information-theoretic efficiency of written language for parallel text data in more than 1000 different languages, both on the level of characters and on the level of words as information encoding units. The main results show that (i) the median efficiency is ∼29% on the character level and ∼45% on the word level, (ii) efficiency on both levels is strongly correlated with each other and (iii) efficiency tends to be higher for languages with more speakers
Arithmetic · Linguistics · Natural language processing · Computer Science · Fractal and DNA sequence analysis · Language and cultural evolution · Machine Learning in Bioinformatics · Mathematics · Artificial Intelligence
Analyzing Linguistic Data
Beta Regression for Modelling Rates and Proportions
Permutation inference for the general linear model
How Efficiency Shapes Human Language
Zipf’s word frequency law in natural language
The indeterminacy of word segmentation and the nature of morphology and syntax
Prediction and Entropy of Printed English
GlmmTMB Balances Speed and Flexibility Among Packages for Zero-inflated Generalized Linear Mixed Modeling
Advanced Bayesian Multilevel Modeling with the R Package brms
A new look at the statistical model identification
A Mathematical Theory of Communication
A Nonstochastic Interpretation of Reported Significance Levels
Information Theory and Redundancy
The evolution of language families is shaped by the environment beyond neutral drift
Revisiting population size vs. phoneme inventory size
| Velocidad de citación | historical |
|---|---|
| Altamente citado | No |