Topic XX. Blind Analysis
Filtering on
Context for this filter:
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Blind analysis: Making all decisions regarding data analysis before the results of interest are unveiled, such that expectations about the results do not bias the analysis. Usually co-occurs with a commitment to publicize the results however they turn out.
- Examples of analysis decisions for which blind analysis could be useful:
- Confirmation bias drives the need for blind analysis.
- Scientists are constantly looking for bugs in scientific practices in order to fix them. Blind analysis is just the latest example of scientists recognizing a bug in their practice (e.g., a way of being fooled) and adjusting practice to account for/remove the bug.
-
EXAMPLES
- Cautionary Quotes: Mistakes, Misconceptions, & Misunderstandings
- Confusion between blind analysis and double blind experiments is common. These concepts are related but distinct.
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Examples of analysis decisions for which blind analysis could be useful:
- a. Stopping rules for when to stop looking for flaws in your experimental design or for computer/math bugs.
- b. Data selection decisions.
- c. Decisions about which analysis procedures to use (e.g. grouping decisions).
- C. CONCEPT APPLICATION
- Explain why blind analysis might be needed, by explaining the errors that might arise in its absence.
- Recognize when blind analysis is being used and explain what function it serves. Identify situations and decisions that would call for blind analysis.
- Evaluate techniques (e.g., registered replication, adversarial collaboration, peer review) - a) for ability to address confirmation bias and b) in comparison to blind analysis.
- Propose how to use blind analysis for simple studies.