Using Bayes to get the most out of non-significant results
Bibliographic Data
| ID | 5771723 |
|---|---|
| Authors | Zoltan Dienes (0000-0001-7454-3161, University of Sussex, corresponding author) |
| Year | 2014 |
| Volume | 5 |
| Publication date | 2014-07-29 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Frontiers in Psychology (JOURNAL) |
| Journal identifiers | ISSN: 1664-1078 • E-ISSN: 1664-1078 |
| Publisher | Frontiers Media (PUBLISHER • CH) |
| DOI | 10.3389/fpsyg.2014.00781 |
| PMID | 25120503 |
| PMCID | PMC4114196 |
| OpenAlex | W2167796542 |
| Language | EN |
| Citations received | 293 |
| References cited | 89 |
No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory's predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors
Advanced Statistical Methods and Models · Bayesian Modeling and Causal Inference · Forecasting Techniques and Applications
A transient effect of episodic simulation in repeated decision-making
Addiction is a brain disease, and it doesn’t matter
A Bayesian perspective on severity
Sensory attenuation of action outcomes of varying amplitude and valence
Effect of mindfulness meditation on brain–computer interface performance
Susceptibility to unconscious influences is unaffected by a challenging inhibitory task or mental exhaustion
Habitual control of instrumental behaviour requires conscious stimulus perception
Target meta-awareness is a necessary condition for physiological responses to masked emotional faces
The effect of facial feedback on the evaluation of statements describing everyday situations and the role of awareness
The role of edge-based and surface-based information in natural scene categorization
Studying human eating behaviour in the laboratory
Bayesian inference for psychology. Part I
Sequential hypothesis testing with Bayes factors
Bayesian inference for psychology. Part II
How Bayes factors change scientific practice
Replication, falsification, and the crisis of confidence in social psychology
Bayes factor design analysis
The Bayesian New Statistics
Bayesian data analysis for newcomers
Indices of Effect Existence and Significance in the Bayesian Framework
Do Student Samples Provide an Accurate Estimate of the General Public?
Assessing the impact of hybrid school visits in initial teacher education
Advancing the Cognitive Science of Religion through Replication and Open Science
Effects of survival processing on list method directed forgetting
Statistical and explicit learning of graphotactic patterns with no phonological counterpart
Empathy in leadership communication
The Relationship of Musical Expertise, Working Memory, and Intelligence
Writing by hand or digitally in first grade
Learning to code and the acquisition of computational thinking by young children
Examining the effect of Libet clock stimulus parameters on temporal binding
Perceptual flexibility in word learning
An ERP study of hemispheric differences in perceptual representations of language reveals meaning attribution in the right hemisphere and constituents of the N400-effect
Evidence that instrumental conditioning requires conscious awareness in humans
The ontogeny of early language discrimination
Interactions between givenness and clause order in children’s processing of complex sentences
Automatic imitation does not predict levels of prosocial behaviour in a modified dictator game
Exploring the potential of a game-based preschool assessment of mathematical competencies
Investigating the relationship between individual differences and island sensitivity
Pilot Study on the Effects of Music-making on Cortisol, Alpha-amylase and Immunoglobulin-a Among Older Amateur Musicians
Socioeconomic conditions exacerbated by the coronavirus in the Eastern Cape province of South Africa
Affecting Explicit and Implicit Body Image with Thin-Idealized Avatars in Virtual Reality
Motivational brief interventions for adolescents and young adults with Internet use disorders
Now You See Me, Now You don’t
Mindsets of poverty
Watching Videos on a Smartphone
Can templates-for-rejection suppress real-world affective objects in visual search
Letter chunk frequency does not explain morphological masked priming
Can I trust this paper
No evidence for association between pupil size and fluid intelligence among either children or adults
Avoiding pitfalls
The Jasp guidelines for conducting and reporting a Bayesian analysis
Distinct but related abilities for visual and haptic object recognition
Is rate-dependent perception affected by linguistic information about the intended syllable rate
Teleporting into walls? The irrelevance of the physical world in embodied perspective-taking
Some neighbors are more interfering
The generality of the attentional boost effect for famous, unfamiliar, and inverted faces
Dispositional mindfulness
Refinement of face representations by exposure reveals different time scales of biases in face processing
Common methodological mistakes
Construal level of traffic violation behavior
How do menstrual cycle and emotional expressions affect attentional and memory biases
Expressing gratitude and visualising one’s best possible self
Exploring mechanisms underlying intertemporal choices
Math anxiety and attention
Differences in dark triad personality
Episodic and semantic memory contributions to imagination and creativity
False memory-guided eye movements
Magnitude and sources of proactive interference in visual memory
Effects of saccadic eye movements on episodic & semantic memory fluency in older and younger participants
The animacy (bias) effect in recognition
The positive dimension of schizotypy is associated with self-report measures of autobiographical memory and future thinking but not experimenter-scored indices
Retrieval-induced forgetting with novel visual stimuli is retrieval-specific and strength- independent
Are changes in binge drinking among European adolescents driven by changes in computer gaming
Dualists and physicalists agree, free will is incompatible with determinism
Towards a processing model for argument-verb computations in online sentence comprehension
Retrieval practice and verbal-visuospatial transfer
Where is mirror invariance? Masked priming effects by mirrored and rotated transformations of reversible and nonreversible letters
Does high variability training improve the learning of non-native phoneme contrasts over low variability training? A replication
Anticipating causes and consequences
Language learning as uncertainty reduction
Risk prediction algorithms and clinical judgment
The Covid generation
No Heightened Musical Pitch Weighting For Tone Language Speakers in Early Childhood
Musical Advantage in Lexical Tone Perception Hinges on Musical Instrument
Exploring boundary conditions of goal-driven attentional capture by affective categories
Not all emotional expressions facilitate recognition of other-race faces in Chinese infants
A differential impact of action–effect temporal contiguity on different measures of response inhibition in the Go\No-Go and Stop-signal paradigms
Probing the mechanisms underlying numerosity-to-numeral mappings and their relation to math competence
Linguistic markedness and body specificity in parity judgments
You do you
“Speak of the Devil… and he Shall Appear”
Positively framing mind wandering does not increase mind wandering in older adult s
Re-examining the effect of motivation on intentional and unintentional task-unrelated thought
An implicit task reveals space-time associations along vertical and diagonal axes
Can teaching assistants improve attainment and attitudes of low performing pupils in numeracy? Evidence from a large-scale randomised controlled trial
Emotional distraction in working memory
Guiding the use of collective feedback displayed on heatmaps to reduce group conformity and improve learning in Peer Instruction
There is Nothing Magical about Bayesian Statistics
That’s Racist
Blinking While Speaking and Talking, Hearing, and Listening
Error and the Growth of Experimental Knowledge
Bayesian Cognitive Modeling
Probability Theory
Bayesian Analysis for the Social Sciences
A practical solution to the pervasive problems ofp values
Optional stopping
Statistical Evidence in Experimental Psychology
Bayes factor approaches for testing interval null hypotheses.
Bayesian Assessment of Null Values Via Parameter Estimation and Model Comparison
An Agenda for Purely Confirmatory Research
Model Selection and Multimodel Inference
Bayesian t tests for accepting and rejecting the null hypothesis
Power failure
Is the Replicability Crisis Overblown? Three Arguments Examined
Bayesian mediation analysis.
Default Bayes factors for Anova designs
Bayesian Versus Orthodox Statistics
Statistical power analyses using G*Power 3.1
Statistical Power Analysis
The speed of metacognition
Explicit feedback maintains implicit knowledge
Alcohol increases hypnotic susceptibility
Subliminal understanding of negation
Effect of mindfulness meditation on brain–computer interface performance
Conscious and unconscious thought in artificial grammar learning
Suggestibility and suggestive modulation of the Stroop effect
Learning without consciously knowing
Unconsciously learning task-irrelevant perceptual sequences
Hypnotic suggestibility predicts the magnitude of the imaginative word blindness suggestion effect in a non-hypnotic context
Rationality in psychological research
The statistical power of abnormal-social psychological research
The earth is round (p < .05)
Things I have learned (so far)
Standardized or simple effect size
Using significance tests to evaluate equivalence between two experimental groups
Do studies of statistical power have an effect on the power of studies
Consequences of prejudice against the null hypothesis
| Unique citing works | 293 |
|---|---|
| Citations per year | 24,42 |
| Citation span | 2014 - 2026 (13) |
| Citation velocity | current |
| Highly cited | Yes |
| Citation types | Neutral: 282 |