Embedding Web-Based Statistical Translation Models in Cross-Language Information Retrieval

Although more and more language pairs are covered by machine translation (MT) services, there are still many pairs that lack translation resources. Cross-language information retrieval (CLIR) is an application that needs translation functionality of a relatively low level of sophistication, since current models for information retrieval (IR) are still based on a bag of words. The Web provides a vast resource for the automatic construction of parallel corpora that can be used to train statistical translation models automatically. The resulting translation models can be embedded in several ways in a retrieval model. In this article, we will investigate the problem of automatically mining parallel texts from the Web and different ways of integrating the translation models within the retrieval process. Our experiments on standard test collections for CLIR show that the Web-based translation models can surpass commercial MT systems in CLIR tasks. These results open the perspective of constructing a fully automatic query translation device for CLIR at a very low cost

Cross-language information retrieval · Embedding · Example-based machine translation · Information retrieval · Language model · Machine translation · Machine translation software usability · Natural language processing · Process (computing · Programming language · Translation (biology · Computer Science · Natural Language Processing Techniques · Semantic Web and Ontologies · Topic Modeling · Artificial Intelligence

• An algorithm for suffix stripping

  Open Access•Martin Porter, M F Porter•Program•1980

  Add to list