Improving Hate Speech Detection via Robust Knowledge Distillation

Hate speech proliferating on social media disrupts the harmony of the Internet, making its detection a challenging task in natural language processing. Despite the recent advances in hate speech detection based on pre-trained language models (PLMs), their large parameter scale limits their applicability on resource-constrained devices, and the substantial noise and data imbalance in online speech severely weaken the model’s ability to extract valid information. To address these issues, we propose a novel robust knowledge distillation framework for improving hate speech detection. This framework aims to leverage the knowledge of a complex teacher model to supervise the training of a compact student model. Specifically, we perform linguistically motivated denoising to mitigate the impact of non-semantic noise on model training, and then transfer the fine-tuned teacher’s knowledge to the student for model compression. To handle data imbalance, we incorporate focal loss into knowledge distillation to enhance the model’s discriminative capability on hard samples. Moreover, we design a dual-round data augmentation strategy to improve model robustness against noisy and perturbations. Experiments on the SE and DV hate speech datasets reveal that our framework boosts the lightweight model’s F1-score by up to 4.3% over the state-of-the-art baseline, while reducing inference time by nearly 40% and resource consumption by 35%–50

Detection theory · Distillation · Filtering theory · Knowledge-based systems · Voice activity detection · Emotion and Mood Recognition · Hate Speech and Cyberbullying Detection · Sentiment Analysis and Opinion Mining