18 <strong>Topic XVII. Pathological Science</strong>
Topic XVII. Pathological Science
- How to catch bad science.
- Distinguishing pathological science, pseudo-science, fraudulent science, poorly-done science, good science that happens to get the wrong answer (which should happen statistically for 1 in 20 papers that give a 0.05 confidence level result). What do practicing scientists do when they try to judge a paper in a field or sub-field outside their immediate area of expertise?
- Addressing the Question: How can we avoid going wrong?
- Bad science and pseudoscience
TOPIC RESOURCES
EXAMPLES
- Exemplary Quotes
- Cautionary Quotes: Mistakes, Misconceptions, & Misunderstandings
- "I took a shot of apple cider vinegar for a Month and just overall felt like it started me on the right track in the morning." https://spoonuniversity.com/lifestyle/i-took-a-shot-of-apple-cider-vinegar-for-a-month-and-i-won-t-stop
LEARNING GOALS
- A. ATTITUDES
- B. CONCEPT ACQUISITION
- The boundaries demarcating science from non-science and distinguishing among the categories of pathological science, pseudo-science, fraudulent science, poorly-done science, and good science can often be difficult, with overlapping and fuzzy boundaries between categories.
- Pathological Science Indicators:
- a. The effect is produced by a barely detectable cause, and the magnitude of the effect is substantially independent of the intensity of the cause.
- b. The effect is barely detectable, or has very low statistical significance. Claims of great accuracy.
- c. Involving fantastic theories contrary to experience.
- d. Criticisms are met with ad hoc excuses.
- e. Ratio of supporters to critics rises to near 50%, then drops back to near zero.
- f. Conclusion-motivated design & analysis.
- g. Pseudo-science is characterized by using scientific vocabulary without aligning with the corresponding concepts or engaging in real scientific practices (i.e., science being “skin deep,” not scientific below the surface).
- Fraudulent science involves intentional deception, such as deliberately fabricating data or deliberately deceiving the reader about the strength of evidence.
- Poorly-done science, e.g. failure to consider confounds, failure to use best practices in terms of data collection and analysis (e.g., small sample size, look elsewhere effect).
- Unintentional self-deception can be involved in justifying poor practices and/or interpretations in pathological, pseudo-, & poorly-done science.
- Motivation to support a particular conclusion (i.e., science undertaken to support a given conclusion, rather than to discover the truth) can be a feature of poorly done or pathological science.
- Good science:
- a. Will get the wrong answer some of the time, e.g., via statistical flukes.
- b. Entails good faith engagement with the alternative hypotheses through a search for evidence that you are wrong.
- C. CONCEPT APPLICATION
- Distinguish science from enterprises such as religion or (perhaps) astrology where the attempt is not to find descriptive adequacy but meaning in ordinary life.
- Identify cases of good science that gets the wrong answer, fraudulent science, pathological science, poorly-done science, and pseudo-science based on the above characteristics.
- Identify what is wrong in cases of fraudulent, pathological science, poorly-done, and pseudo-science.
CLASS ELEMENTS
- Suggested Readings & Reading Questions
- Case Studies (Possible Examples of Pathological Science):
- Wolfe-Simon et al., 2011, "A bacterium that can grow by using arsenic instead of phosphorus."
- Chaplin, 2007, "The Memory of Water."
- Adam et al., "Measurement of the neutrino velocity with the OPERA detectorin the CNGS beam."
- Clicker Questions
- Identifying how many of Langumuir’s 6 Pathological Science indicators are in play in specific papers relating surprising scientice results.
- Discussion Questions
- Class Exercises
- Summaries of three relatively recent surprising science results (e.g. super-luminal neutrinos, bacteria with arsenic in their DNA, water with memory, cold fusion) and their follow-up in the scientific community are distributed among the groups. Each group explains the summary they read to other groups, so all have thought about each example. The groups discuss and vote with clickers on whether each article falls into the category of pathological science, poorly-done science, etc.
- Homework
- Please read the article assigned to you based on your seating chart group (the seating chart for Week 9 is posted on the syllabus). These articles (from a variety of sciences) may be challenging to understand, but we encourage you to try to understand the main ideas from the articles and be prepared to discuss them in class on Wednesday. Please answer the following question for your homework: Based on Langmuir's criteria, do you think that the study conducted in the article you read would qualify as pathological science? In addition to a paragraph explanation summary of your thoughts, make sure to fill out and turn in the “Langmuir Scoresheet” as part of your assignment. [link]