Modeling Information Content Via Dirichlet-Multinomial Regression Analysis

Shannon entropy is being increasingly used in biomedical research as an index of complexity and information content in sequences of symbols, e.g. languages, amino acid sequences, DNA methylation patterns and animal vocalizations. Yet, distributional properties of information entropy as a random variable have seldom been the object of study, leading to researchers mainly using linear models or simulation-based analytical approach to assess differences in information content, when entropy is measured repeatedly in different experimental conditions. Here a method to perform inference on entropy in such conditions is proposed. Building on results coming from studies in the field of Bayesian entropy estimation, a symmetric Dirichlet-multinomial regression model, able to deal efficiently with the issue of mean entropy estimation, is formulated. Through a simulation study the model is shown to outperform linear modeling in a vast range of scenarios and to have promising statistical properties. As a practical example, the method is applied to a data set coming from a real experiment on animal communication

Algorithm · Bayesian probability · Data mining · Dirichlet distribution · Entropy (arrow of time) · Machine learning · Regression analysis · Statistical inference · Statistics · Artificial Intelligence · Bayesian Methods and Mixture Models · Computer Science · Evolution and Genetic Dynamics · Gene Regulatory Network Analysis · Mathematics

• A Simplex Method for Function Minimization

  J A Nelder, R Mead•The Computer Journal•1965

  Add to list