Offensive Language Detection for Low Resource Language Using Deep Sequence Model
Bibliographic Data
| ID | 22107479 |
|---|---|
| Authors | Anas Ali Khan (0000-0002-0964-4748, National University of Sciences and Technology), M Hammad Iqbal (0000-0002-3616-2621, National University of Sciences and Technology), Shibli Nisar (0000-0002-0311-0876, National University of Sciences and Technology), Awais Ahmad (0000-0003-1241-8618, Imam Mohammad ibn Saud Islamic University), Waseem Iqbal (0000-0001-5192-1836, Department of Information Security, National University of Sciences and Technology (NUST), Islamabad, Pakistan) |
| Year | 2024 |
| Volume | 11 |
| Issue | 4 |
| Pages | 5210-5218 |
| Publication date | 2024-08-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | IEEE Transactions on Computational Social Systems (JOURNAL) |
| Journal identifiers | ISSN: 2329-924X • E-ISSN: 2373-7476 |
| Publisher | Institute of Electrical and Electronics Engineers (IEEE) (PUBLISHER) |
| DOI | 10.1109/tcss.2023.3280952 |
| OpenAlex | W4381233115 |
| Language | EN |
| References cited | 46 |
Social media platforms are heavily used by people to express their views in their native languages. Besides positive views, people often use abusive or offensive language to express their anger or frustration. Resource-rich languages have offensive language detection systems which automatically monitor and block offensive content, however, they are very rare for low-resourced languages. This is because of the nonavailability of datasets for local languages. This article proposes a model which automatically detects offensive language for a very low-resource language, i.e., Pashto. The Roman Pashto dataset is created by picking 60 thousand comments from different social media and labeling them manually. The proposed model is trained and tested using three different feature extraction approaches, i.e., bag-of-words (BoW), term frequency-inverse document frequency (TF-IDF), and sequence integer encoding. Four traditional classifiers and a deep sequence model are used to train on this task. Experimental result shows that the random forest classifier works best and give 94.07 % testing accuracy on a combination of unigrams, bigrams, and trigrams. The same classifier gives maximum accuracy of 93.90 % with TF-IDF. However, the overall highest testing accuracy of 97.21% is achieved by using bidirectional long short-term memory (BLSTM). The corpus created in this work is made available for the researcher working in this domain
Bigram · Byte · Language identification · Language model · Machine learning · Natural language · Natural language processing · Offensive · Random forest · Speech recognition · Trigram · Computer Science · Hate Speech and Cyberbullying Detection · Mathematics · Swearing, Euphemism, Multilingualism · Artificial Intelligence
| Citation velocity | historical |
|---|---|
| Highly cited | No |