Probability and confidence
How to improve communication of uncertainty about uncertainty in intelligence analysis
Bibliographic Data
| ID | 12166207 |
|---|---|
| Authors | Misty C Duke (0000-0002-0501-7966, Department of Criminal Justice and Security Studies University of Texas at El Paso El Paso Texas USA, corresponding author) |
| Year | 2023 |
| Volume | 37 |
| Issue | 1 |
| Publication date | 2023-11-27 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Journal of Behavioral Decision Making (JOURNAL) |
| Journal identifiers | ISSN: 0894-3257 • E-ISSN: 1099-0771 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1002/bdm.2364 |
| OpenAlex | W4389069873 |
| Language | EN |
| References cited | 44 |
Intelligence agencies communicate uncertainty to decision‐makers through verbal probability phrases that correspond to numerical ranges (i.e., probability lexicons) and ordinal levels of confidence. However, decision‐makers may misinterpret the relationship between these concepts and form inappropriate interpretations of intelligence analysts' uncertainty. In two experiments, four ways of conveying second‐order probability to decision‐makers were compared: (a) probability and confidence phrases written in the text of a report, (b) the addition of a probability lexicon, (c) the addition of a probability lexicon that varied numerical ranges according to the level of confidence (i.e., revised lexicon), and (d) a probability phrase written in text followed by a numerical range that varied according to the level of confidence. The revised lexicon was expected to improve interpretations of second‐order probability. The 275 participants in Experiment 1 and 796 participants in Experiment 2 provided numerical estimates corresponding to analytic judgments provided in descriptions about three overseas military operations and also indicated their support for approving or delaying the operations. The results demonstrated that providing the numerical range in the text of the report or providing a probability lexicon, improved interpretations of probability phrases above the verbal phrase‐only condition, but not interpretations of confidence. Participants were unable to correctly interpret confidence with respect to the precision of their estimate intervals and their decisions about the operations. However, in Experiments 2 and 3 the effects on these variables of providing decision‐makers with information about the source of the analyst's uncertainty were examined. In Experiment 3 ( n = 510), providing this information improved correspondence between confidence level and approval of the operation. Recommendations are provided regarding additional methods of improving decision‐makers' interpretation of second‐order probability conveyed in intelligence reporting
Confidence interval · Lexicon · Natural language processing · Phrase · Range (aeronautics · Statistics · Advanced Text Analysis Techniques · AI-based Problem Solving and Planning · Computer Science · Intelligence, Security, War Strategy · Mathematics · Artificial Intelligence
The concept of a linguistic variable and its application to approximate reasoning—I
Numeracy and Decision Making
Variants of uncertainty
General Performance on a Numeracy Scale among Highly Educated Samples
Effective communication of uncertainty in the IPCC reports
Fuzzy sets
Uncertainty, Intelligence, and National Security Decisionmaking
Meta‐informational cue inconsistency and judgment of information accuracy
Understanding ‘Unlikely (20% Likelihood)’ or ‘20% Likelihood (Unlikely)’ Outcomes
Analytic Confidence and Political Decision‐Making
Behavioral Consequences of Probabilistic Precision
How numeracy influences risk comprehension and medical decision making
| Citation velocity | historical |
|---|---|
| Highly cited | No |