Half-Context Language Models

This article investigates the effects of different degrees of contextual granularity on language model performance. It presents a new language model that combines clustering and half-contextualization, a novel representation of contexts. Half-contextualization is based on the half-context hypothesis that states that the distributional characteristics of a word or bigram are best represented by treating its context distribution to the left and right separately and that only directionally relevant distributional information should be used. Clustering is achieved using a new clustering algorithm for class-based language models that compares favorably to the exchange algorithm. When interpolated with a Kneser-Ney model, half-context models are shown to have better perplexity than commonly used interpolated n-gram models and traditional class-based approaches. A novel, fine-grained, context-specific analysis highlights those contexts in which the model performs well and those which are better treated by existing non-class-based models

Bigram · Class (philosophy · Cluster analysis · Context (archaeology · Context model · Contextualization · Language model · Machine learning · Natural language processing · Object (grammar · Perplexity · Representation (politics · Trigram · Computer Science · Natural Language Processing Techniques · Speech and dialogue systems · Topic Modeling · Artificial Intelligence

• Exemplar dynamics

  Janet B Pierrehumbert•Frequency and the Emergence of…•2001

  Add to list

• "Schema abstraction" in a multiple-trace memory model.

  Douglas L Hintzman•Psychological Review•1986

  Add to list