Topic I. Role of Science in a Democracy
Filtering on
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Facts vs. Values
- a. Facts: Objectively true claims about reality. Everything that is the case. What is, descriptively, including spatial relations, causal relations, attributes of objects, etc.
- b. Values: What is of value, important, of worth. Oughts, shoulds, etcs.
- Democracy vs. Epistocracy
- a. Democracy: A system of government wherein a society’s citizens have more or less equal input into policies.
- b. Epistocracy: A system of government wherein a particular subset of a society—privileged by their education or other markers of expertise—decides policies.
- C. CONCEPT APPLICATION
- Facts vs. Values
- b. Identify ways in which facts and values are intertwined in public debates and everyday decision-making.
- c. Recognize when values determine which facts are viewed as relevant.
- d. Use distinction between facts and values to identify appropriate source(s) of authority in decision-making.
- Justify when it is appropriate for everyone to have a vote.
- Justify when it is appropriate for science to have a role in democratic decision-making and explain what that role should be.
CLASS ELEMENTS
- Discussion Questions
- If you could create a government from scratch, to what extent and in what way would it be epistocratic vs. democratic? Why?
- How would your system come to a decision on a healthcare plan?
- Utopias:
- A. Imagine a utopia in which democracy functioned optimally (as well as it conceivably could). What would such a society be like? How is it different from our society? How hard would it be to bring about such a society?
- B. Now imagine a utopia in which epistocracy functioned optimally. What would this society be like? How realistic is it?
-
OVERVIEW
- Addressing the Question: Why is Science Effective?
- The Reality Assumption
- Science as Self-Correcting
EXAMPLES
- Exemplary Quotes
- Cautionary Quotes: Mistakes, Misconceptions, & Misunderstandings
- "Science is just another religion, no better and no worse than any other. They use textbooks as their scripture, and scientists are their priests. You should choose whichever authority feels most right to you or stick with the authority you were raised with, because there's no other way to choose between them."
LEARNING GOALS
- A. ATTITUDES
- Appreciate how scientific theories can be trustworthy even though past theories have turned out to be wrong.
- Appreciate how science can be both always subject to challenge, yet (often) rightly depended on for practical decision-making.
- Optimism in capacity of science to help solve problems & improve representations, for both political & personal decision-making.
- B. CONCEPT ACQUISITION
- Assumption of Reality
- Scientists assume an external reality, which is shared by and affects all people and has enough regularity to lend itself to induction. This external reality is what scientists seek to describe accurately.
- Science vs. Priesthood
- Science gains its authority from its self-questioning character, not from the concentrated power of individuals.
- The process of science leads to self-correction through:
- a. Active experimentation & observation
- b. Peer review
- c. Rewards for better theories, even those that contradict current theories
- d. Responsiveness to new evidence/actively open-minded thinking
- e. Replicability and replication
- f. Multiple approaches to each problem allow convergent evidence
- C. CONCEPT APPLICATION
- Defend critiques of science based on its provisionality by appeal to its self-correcting properties
CLASS ELEMENTS
- Discussion Questions
- Suppose there is scientific consensus on an issue, but you have an intuition that runs against that scientific consensus. Imagine you are obliged to advocate one side or the other (at least provisionally).
- A. Under what conditions, if any, should you go with the scientific consensus?
- B. Under what conditions, if any, should you go against the scientific consensus?
-
OVERVIEW
- It has been suggested that our trust in "a reality out there" is often strengthened by our actively interacting with the passively-percieved world (banging the table in front of us with our hand). Our “direct” experience with a scientist’s reality is expanding further: from the human senses to the human armed with instruments. The novel instruments that are now with us constantly (e.g. GPS, camera) allow us to interactively explore parts of the world that until recently were inaccessible or accessible only passively, through expensive technology or images made by scientists. This interactive experience of previously inaccessible aspects of the world now revealed by technology is broadening our sense of what counts as "real." We carry with us a growing range of interactive tools, these days primarily in the compact form of our smartphones.
EXAMPLES
- Exemplary Quotes
- “It’s amazing to first see a slow motion picture of a violin string making a note—wouldn’t it be great if our eyes and brains were fast enough to do this?”
- “Do you think that modern technology offering us more different vantage points fundamentally changes our position on any practical questions? For example, does it make a difference that in relatively recent decades we have gotten used to seeing the earth as a whole from space?”
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Validity of a Measure: Is the measure yielding information about the target entity in out in the world, given that the target is something real (i.e., concept is valid).
- Challenges in validating the use of an instrument:
- a. When there is no objective gold standard (e.g. passage of time, what fluid to use in a thermometer)
- b. When direct observation is messy or impossible (e.g. radio waves, size)
- Techniques for validating instruments:
- a. Interactive exploration: Testing an instrument by changing the thing it is measuring in ways you know through other means, and seeing if the instrument recognizes the changes appropriately (e.g. does driving increase a car's odometer; see how singing higher and lower notes affects a sound spectrograph; sprayable electrons in Hacking reading).
- b. Comparison of multiple instruments (e.g. thermometers)
- c. Comparison to direct observation (e.g. naked sight compared to sight with a magnifying glass)
CLASS ELEMENTS
- Class Exercises
- IPhone app provides interactive sound spectrogram (Spectogram Pro). Use slide whistle, stringed instrument, whistle (interactive exploration high vs. low), difference in timbre between male & female voices. What differences does the spectrograph instrument show between these sounds? How do these differences map onto differences you can hear? What does the spectrograph show that you can't know just by listening? Do you believe that what the spectrograph shows that you can't hear is real? Why or why not?
- iPhone app (Vernier Video Physics) shows quantitative analysis of slices of time after videotaping the movement of a tossed ball (falling and bouncing).
-
OVERVIEW
- We examine sources of statistical uncertainty/error (which can be averaged down) and systematic uncertainty (which can’t). We also connect these concepts to related terminology (jargon) from other fields: precision vs. accuracy, variance vs. bias, and reproducibility vs. validity.
EXAMPLES
- Exemplary Quotes
- "It won't do us any good to average lots and lots of test subjects' heights together if our tape measure got shrunk in the wash!"
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Statistical Uncertainty/Error: Differences between reality and our measurement on the basis of random imprecisions.
- a. All measurements have a certain amount of variance, which are just differences between multiple measurements due to error and/or genuine variation in the sample. These differences will not all go in the same direction.
- b. Statistical uncertainty can be reduced by averaging a larger amount of data.
- Systematic Uncertainty/Error: Differences between reality and our measurement that skew our results in one direction.
- a. Such measurements will show a consistent bias, that is, a consistent deviation from reality in one direction.
- b. Systematic uncertainty cannot be reduced by averaging a larger amount of data.
CLASS ELEMENTS
- Class Exercises
- Students line up in “human histograms,” demonstrating statistical dispersion and systematic bias.
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Scientific Optimism: An attitude of optimism that persistence and iteration on a difficult scientific problem will eventually pay off with interesting insights into your problem.
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Randomized Controlled Trial (RCT): An attempt to identify causal relations by randomly assigning subjects into two groups and then performing an experimental intervention on the subjects in one of the groups.
- a. Experimental Intervention: The act of an experimenter changing one variable in a possible causal network.
- b. Randomized Assignment: Given sufficient sample size, randomized assignment rules out confounds by distributing variation randomly between the two groups, thereby avoiding systematic differences except as the result of the intended intervention.
- c. Control Condition: Comparison of an experimental to a control condition is necessary in order to distinguish effect of intervention from changes that would have occurred without the intervention.
- d. Sampling: A study of a well-chosen sample can tell you something about the population (through induction), especially if it was selected in such a way as to avoid any systematic differences between the sample and the rest of the population. It is often difficult or even impossible to capture a perfectly representative sample, so scientists do the best they can. For example, many psychology studies are done with college students because they are accessible, but such samples differ systematically from the general population. Inferences from samples to a larger population need to take such differences into account.
-
LEARNING GOALS
- A. ATTITUDES
- Appreciate that we can sometimes get very good evidence for a causal hypothesis, even in the absence of decisive RCTs.
- Be wary of potential confounds in apparent evidence for causality.
- B. CONCEPT ACQUISITION
- There are non-RCT forms of evidence for causal hypotheses, which are less conclusive than RCTs but together can offer strong evidence for causation. These include:
- a. Prior plausibility: Can a plausible mechanism be constructed, or is there some other basis for interpreting the current evidence in terms of one causal structure over another, such as data from other studies?
- c. Dose-response curve: Do the quantities of the hypothesized cause correlate with the quantity, severity, or frequency of the hypothesized effect across ranges?
- C. CONCEPT APPLICATION
- For a given causal hypothesis and imperfect study, identify the imperfections (e.g., sample size, lack of randomization, lack of control) and explain how these imperfections impact claims of causality.
- Identify potential confounds in RCT and non-RCT studies.
- For a given scenario in which a causal hypothesis/claim is being made, identify plausible alternative hypotheses that could be consistent with the data.
- Identify additional sources of evidence that could be used to help mitigate flawed experiments, including prior plausibility, dose-response relationships, size of effect, temporal ordering, and multiple complementarily-flawed experiments.
-
OVERVIEW
- Addressing the Question:How do we find out how things work?
- Singular vs. General Causation
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Singular Causation: A causal relation between specific events — i.e., Event A caused Event B.
- General Causation: A causal relation between variables — i.e., X causes Y.
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Signal-to-Noise Ratio: The relative strength of signal compared to the relative strength of noise in a given context. Obtaining meaningful information from the world requires distinguishing signal from noise. Therefore, human cognition (both scientific and otherwise) relies on techniques and tools to suppress noise and/or amplify signal (i.e., increase signal-to-noise ratio).
- It is possible to design filters to increase the signal-to-noise ratio, if you know where the noise is going to appear.
- C. CONCEPT APPLICATION
- Describe examples of techniques and tools to suppress noise and/or amplify signal (i.e., increase signal-to-noise ratio).
CLASS ELEMENTS
- Class Exercises
- Playing a sound with Morse code signal hidden in static. Demonstrate how our ear/brain is highly developed to find the signal.
- Students write down a short phrase that they proceed, by stages, to hide in more and more noise (random substituted letters). Show the concept of “signal-to-noise ratio” as away to quantify at what point they can no longer recognize the message (the signal).
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- False Positive/Type I Errors: A test yields a positive result, but in fact the condition is not present.
- False Negative/Type II Errors: A test yields a negative result, but in fact the condition is present.
- There is always the possibility of a trade-off—for a given test, one can reduce the risk of false positives by increasing the risk of false negatives, and vice-versa.
- Good decision-making under uncertainty involves having sufficient signal (an adequate test) and setting your threshold appropriately for the relative costs of false positives and false negatives.
- In some classification cases like pornography identification or graduate school admissions, there may not be a “truth of the matter” so there aren’t true “false” positives or “true” negatives, although a threshold must still be set.
- C. CONCEPT APPLICATION
- Weigh the costs associated with false positives/negatives with the benefits associated with true positives/negatives when making a decision under uncertain conditions.
- Explain how people could come to different decisions or policies as a result of different utilities/values associated with different types of errors, even if they agree about the relevant facts.
-
OVERVIEW
- An important element of the culture of science is the use of “tentative” propositions, often quantified. These can be as confident as 99.99999%—you would bet your life on it—but it would still be understood to be held as a proposition which could be wrong. This makes it psychologically easier for a scientist to be open to being wrong—and to look actively for ways they might have gotten it wrong. This cultural understanding of the importance of recognizing and reporting one's credence level leads to insistence on including error bars on graphs: a data point is completely meaningless without an error bar.
- Addressing the Question: How confident should we be?
- Credence/Confidence Levels #mq
- Calibration of Credence/Confidence Levels
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Credence: level of confidence that a claim is true, from 0 to 1.
- Accuracy: How frequently one is correct; proximity to a true value.
- Calibration: How closely confidence and accuracy correspond; that is, how accurate a person or system is at estimating the probability that they are correct.
- Because every proposition comes with a degree of uncertainty:
- a. Partial and probabilistic information still has value.
- d. Scientific culture primarily uses a language of probabilities, not certain facts.
- e. Even correctly-done science will obtain incorrect results some of the time.
CLASS ELEMENTS
- Class Exercises
- A arbitrary topic is chosen for small group discussion (e.g. “Does testing in the schools help or hurt education?”), but during the discussion the students have to state their credence level (by saying a number between 0 and 100%) after every statement that they make which could have a credence level associated with it.
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Multiple Causation: Any given effect may be brought about by a complex combination of many causes (which may interact with each other), with varying degrees of influence on the outcome.
- C. CONCEPT APPLICATION
- Identify multiple causes for a given effect or outcome and evaluate the relative importance of each with respect to different contexts and goals.
CLASS ELEMENTS
- Clicker Questions
- The final such question for this class concerns parsing some of the biggest drivers of government spending, specifically what are the relative “orders” of the contributions to total government spending from the costs of education, incarceration, and social security.
-
CLASS ELEMENTS
- Class Exercises
- Together with the whole class, the professor shows how to develop Fermi-problem estimates of a given quantity, e.g. the amount per year that Americans spend on gas for personal transportation.
- In small groups, students use Fermi estimates to re-think the first-order, second-order, etc. parsing of US government spending on education, incarceration, and social security (following up the final activity from Topic XIIV, Orders of Understanding). The students frequently reach completely different orderings than they did in the previous class—and come within ~20% of the actual amounts spent.
-
LEARNING GOALS
- A. ATTITUDES
- Recognize the potential to abuse science for social and political ends.
- B. CONCEPT ACQUISITION
- Just World Fallacy: The tendency to believe that outcomes are deserved and existing social structures are justified.
-
CLASS ELEMENTS
- Class Exercises
- Students answer typical wisdom-of-crowd estimate questions using their clickers—but they can update their estimates as they see the histogram with the other students’ guesses. Afterwards, it is shown that the accuracy of the class’ mean estimate actually got worse as they continued to update their estimate—showing (if it works) that wisdom of crowd works best if the inputs are independent.
-
LEARNING GOALS
- B. CONCEPT ACQUISITION
- Deliberative Polling: A system of decision-making wherein a representative group of stakeholders come together, exchange and discuss information, question experts, and poll their individual opinions at various points throughout the session.
- C. CONCEPT APPLICATION
- Recognize problems which could be resolved effectively using deliberative polling.
- Plan deliberative polls, including:
- a. Identifying relevant stakeholders.
- b. Identifying relevant experts.
- c. Identifying important types of factual information & questions.
- d. Proposing appropriate questions to ask in the polls.
CLASS ELEMENTS
- Class Exercises
- In the two classes of this week, we try out an approach to random-sample-representative decision-making, using a panel of experts to answer questions generated by small deliberative groups, each with a moderator.