A Local Self-Attention Sentence Model for Answer Selection Task in CQA Systems
Bibliographic Data
| ID | 22108408 |
|---|---|
| Authors | Donglei Liu (0000-0002-7779-3550, Beijing Institute of Technology), Hao Lu (0000-0001-7020-6854, Chinese Academy of Sciences), Yong Yuan (0000-0003-4528-3879, Renmin University of China), Rui Qin (0000-0003-3473-2173, Chinese Academy of Sciences), Yifan Zhu (0000-0002-8195-5559, Beijing Institute of Technology), Chunxia Zhang (0000-0003-4805-1109, Beijing Institute of Technology), Zhendong Niu (0000-0002-0576-7572, Beijing Institute of Technology) |
| Year | 2023 |
| Volume | 10 |
| Issue | 6 |
| Pages | 3283-3294 |
| Publication date | 2023-12-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | IEEE Transactions on Computational Social Systems (JOURNAL) |
| Journal identifiers | ISSN: 2329-924X • E-ISSN: 2373-7476 |
| Publisher | Institute of Electrical and Electronics Engineers (IEEE) (PUBLISHER) |
| DOI | 10.1109/tcss.2022.3197652 |
| OpenAlex | W4293173889 |
| Language | EN |
| Citations received | 1 |
| References cited | 35 |
Current evidence indicates that the semantic representation of question and answer sentences is better generated by deep neural network-based sentence models than traditional methods in community answer selection tasks. In particular, as a widely recognized language model, the self-attention model computes the similarity between the specific word and the whole sets of words in the same sentence and generates new semantic representation through the similarity-weighted summation of semantic representations of the whole words. However, the self-attention operation entirely considers all the signals with a weighted sum operation, which disperses the distribution of attention, which may result in overlooking the relation of neighboring signals. This issue becomes serious when applying the self-attention model to online community question answering platforms because of the varied length of the user-generated questions and answers. To address this problem, we introduce an attention mechanism enhanced local self-attention (LSA), which restricts the range of original self-attention by a local window mechanism, thereby scaling linearly when increasing the sequence length. Furthermore, we propose stacking multiple LSA layers to model the relationship of multiscale $n$ -gram features. It captures the word-to-word relationship in the first layer and then captures the chunk-to-chunk (such as lexical $n$ -gram phrases) relationship in its deeper layers. We also test the effectiveness of the proposed model by applying the learned representation through the LSA model to a Siamese and a classification network in community question answer selection tasks. Experiments on the public datasets show that the proposed LSA achieves a good performance
Natural language processing · Sentence · Advanced Text Analysis Techniques · Computer Science · Engineering · Sentiment Analysis and Opinion Mining · Topic Modeling · Artificial Intelligence
| Citations per year | 0.5 |
|---|---|
| Citation span | 2024 - 2024 (1) |
| Highly cited | No |