Topic IX. Seeing Patterns in Random Noise
Filtering on
Context for this filter:
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Look Elsewhere Effect: Even if there is a low probability of pure noise passing a given threshold for signal, if we look at enough noise some of it will pass that threshold by chance. I.e., if there is a low probability of obtaining a false positive in any given instance, the more times you try (the more questions you ask, measures you take, or studies you run without statistical correction), the more you increase the probability of getting a false positive. This occurs when one:
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- False Positive/Type I Errors: A test yields a positive result, but in fact the condition is not present.
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Look Elsewhere Effect: Even if there is a low probability of pure noise passing a given threshold for signal, if we look at enough noise some of it will pass that threshold by chance. I.e., if there is a low probability of obtaining a false positive in any given instance, the more times you try (the more questions you ask, measures you take, or studies you run without statistical correction), the more you increase the probability of getting a false positive. This occurs when one:
- a. Asks too many questions of the same data set, reporting only statistically significant results.
- b. Asks the same question of multiple data sets, reporting only statistically significant results.
- c. Runs a test or similar tests too many times, reporting only statistically significant results.
- d. This also occurs in everyday life, e.g. when one looks at a whole lot of phenomena and only takes note of the most surprising-looking patterns, not properly taking into account the larger number of unsurprising patterns/lack of pattern.
-
EXAMPLES
- Exemplary Quotes
- "The oncoming asteroid has only a 1% chance of hitting Earth. But if it does, life on Earth will be destroyed. It'll be expensive to stop the asteroid, but the risk is bad enough it's worth it."
- Cautionary Quotes: Mistakes, Misconceptions, & Misunderstandings
LEARNING GOALS
- C. CONCEPT APPLICATION
- Identify false positive (type I) and false negative (type II) errors in scientific and everyday situations.
- Weigh the costs associated with false positives/negatives with the benefits associated with true positives/negatives when making a decision under uncertain conditions.
- Explain how people could come to different decisions or policies as a result of different utilities/values associated with different types of errors, even if they agree about the relevant facts.