Entropy and Uncertainty

Bibliographic Data

ID10706014
AuthorsTeddy Seidenfeld (0000-0003-4828-2388, Carnegie Mellon University, corresponding author)
Year1986
Volume53
Issue4
Pages467-491
Publication date1986-12-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenuePhilosophy of Science (JOURNAL)
Journal identifiersISSN: 0031-8248 • E-ISSN: 1539-767X
PublisherCambridge University Press (CUP) (PUBLISHER)
DOI10.1086/289336
OpenAlexW4243742020
LanguageEN
Citations received14
References cited18

This essay is, primarily, a discussion of four results about the principle of maximizing entropy (MAXENT) and its connections with Bayesian theory. Result 1 provides a restricted equivalence between the two: where the Bayesian model for MAXENT inference uses an “a priori“ probability that is uniform, and where all MAXENT constraints are limited to 0–1 expectations for simple indicator-variables. The other three results report on an inability to extend the equivalence beyond these specialized constraints. Result 2 established a sensitivity of MAXENT inference to the choice of the algebra of possibilities even though all empirical constraints imposed on the MAXENT solution are satisfied in each measure space considered. The resulting MAXENT distribution is not invariant over the choice of measure space. Thus, old and familiar problems with the Laplacian principle of Insufficient Reason also plague MAXENT theory. Result 3 builds upon the findings of Friedman and Shimony (1971; 1973) and demonstrates the absence of an exchangeable, Bayesian model for predictive MAXENT distributions when the MAXENT constraints are interpreted according to Jaynes's (1978) prescription for his (1963) Brandeis Dice problem. Lastly, Result 4 generalizes the Friedman and Shimony objection to cross-entropy (Kullback-information) shifts subject to a constraint of a new odds-ratio for two disjoint events

Bayesian probability · Econometrics · Entropy (arrow of time) · Kullback–Leibler divergence · Principle of maximum entropy · Statistics · Computer Science · Decision-Making and Behavioral Economics · Mathematics · Philosophy and History of Science · Statistical Mechanics and Entropy