Testing language models’ syntactic sensitivity to grammatical constraints
A case study of wanna contraction
Bibliographic Data
| ID | 22092390 |
|---|---|
| Authors | Kangsan Noh (Korea University), Eunjeong Oh (Sangmyung University, corresponding author), Sanghoun Song (0000-0002-4234-232X, Korea University) |
| Year | 2024 |
| Volume | 9 |
| Publication date | 2024-12-19 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Frontiers in Communication (JOURNAL) |
| Journal identifiers | ISSN: 2297-900X • E-ISSN: 2297-900X |
| Publisher | Frontiers Media SA (PUBLISHER • CH) |
| DOI | 10.3389/fcomm.2024.1442093 |
| OpenAlex | W4405609191 |
| Language | EN |
| References cited | 18 |
Wanna contraction refers to the reduction of want to to wanna . Interestingly, native English speakers contract want to in object extraction questions but not in subject extraction questions. The present study investigated whether language models such as bidirectional encoder representations from transformers (BERT) adhere to this grammatical subtlety. Wanna contraction involves two factors: subject–object asymmetry and contraction. Disentangling these two ensures that when language models accurately identify illicit instances of wanna contraction, the detection stems from their understanding of the contraction, rather than the intervention by subject–object asymmetry. For this objective, we conducted three independent experiments. We tested whether language models detect illicit cases of contraction by maintaining constant contraction (Experiment 1) and question types (Experiment 2). We predicted that higher surprisal values would be assigned to ungrammatical instances. The overall results of the two experiments were in line with our prediction (87.5 and 75%, respectively). In addition, the analysis of by-word surprisal also indicates that the models generate higher surprisal values for subject extraction questions in illicit wanna instances (Experiment 3). Thus, the models’ processing patterns of wanna contraction turn out to be close to those of native English speakers, suggesting their role as a research tool in linguistic experiments
Grammar · Linguistics · Natural language processing · Computer Science · Natural Language Processing Techniques · Neurobiology of Language and Bilingualism · Philosophy · Psychology · Syntax, Semantics, Linguistic Variation · Artificial Intelligence
Parsing Effects in Second Language Sentence Processing
Rules and representations
Expectation-based syntactic comprehension
Syntactic Structures
Syntactic Structure from Deep Learning
Acquisition of the constraints on wanna contraction by advanced second language learners
Investigating “wanna” contraction through an emergentist approach among Iranian EFL learners applying usage-based model of language acquisition
Want-to contraction in second language acquisition
Global Rules
| Highly cited | No |
|---|