Filtering on

    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
          • Reading Question: What are the major limitations of democracy? What are its affordances (that is, what functions or goals can it serve well)?
      • Clicker Questions
        • You're on a hike in Marin with your friends and you black out. Next thing you know, you're in an ambulance racing towards the hospital. When you arrive at the hospital, the doctors tell you that one of two things is wrong with you, and they aren't certain which. Either (1.) You are going to die in the next few hours unless you agree to an invasive, dangerous heart surgery, or (2.) You'll probably be fine with some medication, and there's plenty of time to get more tests over the next few days. How do you decide whether to get the heart surgery or take the medication?
          • a. Go with the judgment of the most experienced doctors at the hospital.
          • b. Get everyone in the hospital, patients, visitors, doctors, nurses, janitors, and your worried friends, to vote on 1 or 2, and go with the majority vote.
        • You're the school president at a new school, and you are in charge of setting up the process to choose a school mascot. Students have already proposed a list of twenty possibilities. Which is the best way to make this choice?
          • a. Invite in an external historian of school mascots to choose, since they are an expert.
          • b. Ask the teachers and administrators to vote, since they know more than the students.
          • c. Give one vote token to every person in the school, teachers and students, and go with the most popular.
        • Discuss: What's the difference between questions 1 (what to do about a heart attack) and 2 (how to choose a school mascot)?
        • If you had to choose between living in a pure democracy or a pure epistocracy, which would you choose?
          • A. Democracy
          • B. Epistocracy
      • Discussion Questions
        • Why did you give the answer you did to the clicker question (better to live in a pure democracy or a pure epistocracy)? What are the costs of your choice?
          1. In what ways is the current US political system epistocratic and democratic?
          1. If you could create a government from scratch, to what extent and in what way would it be epistocratic vs. democratic? Why?
          • How would your system come to a decision on a healthcare plan?
          1. Utopias:
          • A. Imagine a utopia in which democracy functioned optimally (as well as it conceivably could). What would such a society be like? How is it different from our society? How hard would it be to bring about such a society?
          • B. Now imagine a utopia in which epistocracy functioned optimally. What would this society be like? How realistic is it?
          • C. Would you prefer to live in the democratic utopia or the epistocratic utopia? Why?
          1. How is disagreement about matters of fact different from disagreement about matters of value? How does disagreement about each typically play out? Why are they different?
      • Class Exercises
      • Practice Problems
        • Which of the following is a statement of facts and which of value? Discuss.
          • Socrates is mortal. (F)
          • Everyone should learn a little logic. (V)
          • America is the greatest country on earth. (V, unless "great" is interpreted to mean "powerful," in which case F could be argued.)
          • Dogs have four legs. (F)
          • Trout are the best fish. (V)
          • Everybody loves trout. (F, albeit false)
          • Opera is valuable to society. (V)
          • Soap operas affect social norms. (F)
          • Democracy is the worst system of government, except for all the others. (V)
          • Capital punishment is morally permissible. (V)
          • Capital punishment is an effective deterrent for crime. (F, albeit debatable)
          • We do not protect the oceans well enough. (V, although very easy to argue on the basis of self-interest)
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • ‘Autism’ should be defined:
          • a. Operationally, in terms of observable symptoms. There could be different, equally good definitions.
          • b. Conventionally, by arbitrarily fastening onto one set of symptoms.  
          • c. There’s a real phenomenon out there,‘autism’, but room for continuous progress indefining it. 
        • Suppose there is such a condition as autism 'out there.' Who should be deciding whether it's a disorder or merely part of unproblematic neurodiversity?
          • a. Psychiatrists
          • b. People diagnosed with autism spectrum disorder
          • c. Everyone
          • d. Some other group
      • Discussion Questions
        • If every belief  were truly just as good as any other, what implications would that have for...
          • A. How we should reason about what to believe?  
          • B.  How we should reason about what to do?  
          • C. Human communication?  
          • D. What could be meant when someone calls a claim “true”?  
        • If there were not a shared reality, what would that mean for...
          • science?  
          • group decision-making?
          • communication?
        • Can you think of other epistemic frameworks that, like science, are self-correcting? If so, how are they similar? How different?  
        • What differentiates science from a religion? Describe two elements of science that are not true of religion.
        • Suppose there is scientific consensus on an issue, but you have an intuition that runs against that scientific consensus.  Imagine you are obliged to advocate one side or the other (at least provisionally).  
      • Class Exercises
      • Homework
        • Why wouldn't it be possible to throw out all our beliefs and start completely from scratch?
        • We have mental representations of all kinds of entities that we have never observed with our naked senses, like microbes, the rings around Jupiter, and black holes. How are our representations of these things different from our representations of directly observed entities like kittens and mangoes?
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • Which statement best captures your stance:
          • A. I strongly feel the pull of the arguments about limits of our understanding of reality.
          • B. I strongly feel their pull, but I think there is a little room for our understanding of reality to be essentially correct some of the time.
          • C. I strongly feel their pull, but I am still somehow pretty sure our basic sense of reality is generally right in many of the most important ways.
          • D. I’m strongly confident of our ever-growing capacity to capture reality, almost all of it.
      • Discussion Questions
        • How do we know we can rely on our senses? How do we know when we can't?  
        • How can we observe a thing that we can never perceive directly with our senses?  
        • Describe an entity which you believe exists for which you have only very indirect evidence. Why do you believe it exists? Is there anything that might convince you it did not exist? Is there anything that might convince you that although something like it does exist, it has quite different properties than you had thought? Possible answers: electrons, quarks, black holes, dark matter, souls, God, right and wrong, soulmates, etc .  
        • Suppose you were working for an extremely eccentric art collector. He asks you to measure the beauty of each item in his collection in a systematic way which is not reducible simply to your opinion. How would you go about operationalizing beauty?  
      • Class Exercises
        • CO2 Meter
          • Requires a Windows Computer
          • Blow into CO2 meter to demonstrate that CO2 in the room can be "interactively explored."
        • Diffraction-grating glasses
        • IPhone app provides interactive sound spectrogram (Spectogram Pro). Use slide whistle, stringed instrument, whistle (interactive exploration high vs. low), difference in timbre between male & female voices. What differences does the spectrograph instrument show between these sounds? How do these differences map onto differences you can hear? What does the spectrograph show that you can't know just by listening? Do you believe that what the spectrograph shows that you can't hear is real? Why or why not?
        • iPhone app (Vernier Video Physics) shows quantitative analysis of slices of time after videotaping the movement of a tossed ball (falling and bouncing).
        • A contrasting example (i.e., without much of the interactivity needed to make the "reality" evident): CO2 concentration in room is measured over the course of the class, and the resulting graph of CO2 over time is shown on the projector. You should see a significant increase. Graph should indicate increasing levels of CO2 throughout the class, as the students filling the room fill it with CO2.#h
        • Give instructions to find level function on iPhone (this is unlabeled). Ask students to figure out what it does. Discuss how they know that's what it does.
      • Practice Problems
        • You have three meat thermometers, and they all give different temperatures for your holiday turkey. How would you go about deciding which thermometer to trust?
        • You get a fancy new telescope for looking at stars too distant to see with the naked eye. How would you go about testing if it is showing you real celestial bodies, and not just artifacts of the telescope?
        • Play some more with the spectroscope on your phones. How do you know it is really showing you sounds, and not responding to something else internal or in the environment? Hint: Interactive exploration!!!
    • CLASS ELEMENTS

      • Suggested Readings
      • Clicker Questions
        • Sara is running a mile, which is a four-lap race around a track. Her coach wants to time how fast she runs on the third lap of the four-lap race. As Sara goes past the line on the track that marks the the start of the laps, her coach presses a button on a stopwatch to start a timer. When Sara goes past that line again, her coach presses the button again to stop the stopwatch. Which of the following will be a noticeable source of uncertainty in the coach’s measurement of how long it took Sara to run the lap?
          • A. Primarily statistical uncertainty
          • B. Primarily systematic uncertainty
          • C. Both statistical and systematic uncertainty
          • D. Neither statistical nor systematic uncertainty
        • You have heard that a cat can run up to 30 miles per hour and alligators can move at 11 miles per hour on land, and you want to know how fast you can move. Your friend has the fully automatic electronic timing system that is used for the Olympics, and you use it to time how fast you can run 100 meters at full speed. Your friend fires a starter’s pistol in the air to start your sprint, and the timer automatically starts at the sound of the pistol. The timer automatically stops when you cross a beam of light shining across the finish line. Which of the following will be a noticeable source of uncertainty when you compare your speed to the Olympic record?
          • A. Primarily statistical uncertainty
          • B. Primarily systematic uncertainty
          • C. Both statistical and systematic uncertainty
          • D. Neither statistical nor systematic uncertainty
        • Vote for a candidate for the Berkeley School Board Director
          • A. Judy Appel
          • B. Norma Harrison
          • C. Tracy Hollander
          • D. Jane Shelton
          • E. Beatrice Cutler
      • Discussion Questions
        • How could we find out if there were a systematic bias in our senses?
        • Consider the problem of measuring your class' understanding of types of uncertainty. What is a possible source of systematic uncertainty in measuring each individual's true understanding? What is a possible source of statistical uncertainty? How could both be reduced? Hint: A larger number of and more varied problems on quizzes What would be the cost of reducing these uncertainties in this case? Hint: Too much time spent on quizzes
      • Class Exercises
        • Students line up in “human histograms,” demonstrating statistical dispersion and systematic bias.
        • Small group discussions, clicker-questions, and exercises to have students identify the statistical versus the systematic uncertainties in a number of scenarios. 
      • Practice Problems
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • What is the second-longest that you have ever spent trying to solve a problem/puzzle?
      • Discussion Questions
        • What are the potential costs of insufficient scientific optimism?  
        • What are the potential costs of excessive scientific optimism?  
        • Consider how science works to gradually advance our understanding of the world. Describe how one feature of the scientific process could be usefully applied to policy-making (e.g. scientific optimism, peer review, etc.). Make sure to explain how your suggestions could improve policy-making processes. or
      • Class Exercises
        • Spinning cylinders: A challenging puzzle (involving spinning a piece of plastic tubing, with markings on it) is presented to the students. The experimental conditions end up giving experiential demonstration of the usefulness of scientific optimism.  
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • Suppose there’s an epidemic of Lyn’s disease and a new drug is proposed as a treatment. 100 (or 10,000) patients with the disease are given the drug, and 79 (or 8,700) of them recover. Does this result:
          • A. Give no info about the presence or absence of a causal link
          • B. Establish that the treatment makes no difference.
          • C. Tentatively confirm the efficacy of the treatment, though more evidence may be needed.
          • D. Demonstrate conclusively the existence of an effect.
      • Discussion Questions
        • Why are humans so interested in the causal structure of the world? Hint: It's essential for controlling one's environment.
        • Why are false beliefs about causality so common? E.g. most superstitions are about causality, like astrology, spells, curses, the feeling that you can sometimes will a traffic light to change, etc.
        • Why does randomized assignment to conditions matter in an RCT?
        • Why do RCTs need control conditions?
      • Class Exercises
        • Online Exercise: Causality Lab   
        • Online Exercise: Try five practice rounds at Guess the Correlation. Then refresh and test your average error based on the next five rounds. http://guessthecorrelation.com/  
      • Homework Question
        • Try guessing correlations at http://guessthecorrelation.com/ (Links to an external site.). Once you guess one within .10 of the right answer, screenshot it and upload it.
        • If, among our population of zoo animals, the frequency of ear infections has a -.8 correlation with size, can we infer that ear infections cause creatures to get smaller? Why or why not?
      • Practice Problems
        • Suppose you want to find out whether pumpkins grow bigger when they are given mineral-rich water from a special well. Describe all essential features of a randomized controlled trial that would allow you to determine with reasonable confidence whether or not this is the case.
    • CLASS ELEMENTS  

      • Suggested Readings & Reading Questions
      • Clicker Questions
      • Discussion Questions
        • Brainstorm in small groups three causal links that you are pretty confident are real without RCT evidence. Why are you confident about these causal links? Hint: Think about causal links in everyday life, like allergies or breaking a glass.
      • Practice Problems
        • Observing that major rainstorms tend to develop on the edge of massive cold fronts, scientists conjecture that the cold fronts cause the storms. Give an alternative hypothesis that could explain the data. Then say what evidence could help rule out the alternative hypothesis, and how convincing it would be.
          • Alternative Hypothesis:
          • Distinguishing Experiment:
            • http://ww2010.atmos.uiuc.edu/(Gh)/guides/mtr/af/frnts/cfrnt/prcp.rxml
      • Class Exercises
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
        • No Reading
      • Clicker Questions
        • After a pilot study in which we gave Vertuzi to 100 patients and found 72 of them recovered, we repeat the trial with 100,000 patients, and this time we find that over 82,000 of them recover.
          • A. Give no information about the presence or absence of a causal link
          • B. Establish that the treatment makes no difference
          • C. Tentatively confirm the efficacy of the treatment, though more evidence may be needed
          • D. Demonstrate conclusively the existence of a link
        • We are morally assessable only to the extent that what we are assessed for depends on factors under our control. Two people ought not to be morally assessed differently if the only other differences between them are due to factors beyond their control.
          • A. Agree
          • B. Disagree
        • X causes Y implies that ‘If there was an intervention on X, there would have been a difference to whether Y happened’
          • A. Yes
          • B. No
      • Discussion Questions
      • Practice Problems
        • For each of the following, say whether it is a case of singular causation or general causation; and productive causation or dependent causation; or if there is no causal link. A child is in the street, and a car is coming.
          • A dog jumps into the street and pushes the child out of the way of the car. The child is unhurt, but the dog is killed.
            • A. Car comes --> Dog dies. Singular, Productive
            • B. Child runs --> Dog dies. Singular, Dependent
            • C. Car comes --> Child survives. No causal link
          • Working dogs are very helpful to humans. Police dogs sniff bombs and drugs, seeing eye dogs help blind humans get around, and herding dogs keep sheep or goats from wandering off or getting eaten. During the recent fires in Santa Rosa, a Great Pyrenees called Odin refused to leave his seven goats to get into the car when the fires were coming. His human family had to leave him behind, sure that he and his goats would perish. When they returned, their home and the land around it were in ashes. But Odin came running up. His paws were burnt, but he had not lost a single goat. Somehow two fawns had joined his herd, and he was protecting them, too. News Story
            • A. Herding dogs --> Safety of goats. General, Productive
            • B. Santa Rosa Fires --> Fawns join Odin's flock. Singular, Dependent.
            • C. Fire --> Getting burnt. General, Productive.
            • D. Santa Rosa fires --> Odin's paws burnt. Singular, Productive.
      • Class Exercises
        • Online exercise: Causality Lab
        • Discuss sand resonance demo: https://www.youtube.com/watch?v=wvJAgrUBF4w&t=70s
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • Students identify “noise” and “signal” in different situations (the same event can be “signal” or “noise”depending on context).
        • You are watching a movie on TV. Which of the following is “noise” to you as a movie-watcher?
          • A. A loud crash as a brick flies in the window into the room where the hero is standing.
          • B. An emergency announcement interrupting the movie to warn of an approaching wildfire.
          • C. The pea-soup fog in a scene where the hero is feeling his way through the desolate woods.
          • D. A dramatic political speech by the hero making an important point about democracy.
          • E. All of the above.
        • Same items as above, but now: Which of the following is “noise” from the point of view of the hero?
      • Discussion Questions
      • Class Exercises
        • Playing a sound with Morse code signal hidden in static. Demonstrate how our ear/brain is highly developed to find the signal.  
        • Students write down a short phrase that they proceed, by stages, to hide in more and more noise (random substituted letters). Show the concept of “signal-to-noise ratio” as away to quantify at what point they can no longer recognize the message (the signal).  
        • Play the game Telephone with loud music on, and with silence. In which does the message change the least? (i.e., in which case does the "noise" make the signal harder to keep track of?).
        • Visual version: Handwrite a sentence in pencil. One at a time, each student copies what they think it says, then adds three lines somewhere in the text that alter the letters. By the end, it should be almost impossible to read. Do this with a random sentence, and with a famous sentence (e.g. “We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America”). The famous sentence should be easier to read because it is familiar; we can more easily recognize the pattern.  Contrast with a nonsense line with strange words, e.g. some lines by Lewis Carroll students are not likely to recognize.  Demonstrates that it’s harder to detect a pattern when it’s different from the patterns we’re most used to finding.
      • Practice Problems
        • When using iTunes “shuffle” feature, each song is played only once. However, if you turn shuffle on and off, the order is reshuffled each time, independently of the shuffle before. Consider the following user complaint :
          • i swear i hear the same bands and songs over and over on shuffle. i've got over 4000 songs on my iphone and there's bands on there i never hear while there's bands and songs i hear every single day.https://forums.macrumors.com/threads/is-shuffle-really-random.1133358/
        • Is the recurrence of certain bands and the exclusion of others proof that shuffle isn’t random? Why or why not?
        • Originally, within any particular shuffle you were just as likely to get songs in any order as any other. So if you had 101 songs on the playlist, and the first one was from Dark Side of the Moon, a given different song from Dark Side of the Moon would have a 1/100 chance of being played next. Users complained when getting multiple songs off the same album in a row that shuffle wasn’t really creating a random order. Were they correct to complain? That is, did they have reason to think the order wasn’t random? Why or why not?
        • In response to user complaints, Apple changed the shuffle algorithm to make it less likely you’d hear two songs from the same album in a row. So if you had 101 songs on your playlist and the first one was from Dark Side of the Moon, a given different song from Dark Side of the Moon would have significantly less than a 1/100 chance of being played next. Suppose users complained that the new order (which is still what Apple uses) wasn’t random. Would they have been correct to complain? That is, did they have reason to think the order wasn’t random? Why or why not?
      • Homework
        • Give a new example of a signal that you might be trying to detect (that is, an example not mentioned in the reading). Give an example of noise that might interfere with your detection of this signal. Explain why this is noise. Finally, explain something you could do to minimize the effect of this noise on your signal.
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • Students identify Look Elsewhere Effect mistakes in various scenarios (including medical studies). 
        • A friend tells you that, when conducting coin flips, there were ten heads in a row. This is, of course, a surprising result. In what situation would it be most surprising? 
          • A. She is the only person flipping a coin, and she flipped it only ten times.
          • B. There are many people flipping coins, and everyone flips ten times.
          • C. There are many people flipping coins, and each person conducts 100 flips.
          • D. She is the only person flipping a coin, and she conducts 100 flips.
      • Discussion Questions
        • Students discuss Look Elsewhere Effect mistakes in various scenarios, with Clicker Questions.
        • Listen to excerpt from Radiolab, "A Very Lucky Wind" (first part of their episode on Stochasticity) and discuss.
      • Practice Problems
        • Discuss: https://xkcd.com/882/
      • Class Exercises
        • Professor leaves room and students write down two lists of 40 coin-toss results: “heads, tails, tails, heads...,” the first generated by students sequentially calling out “heads” or “tails,” trying to simulate random coin flips and the second by actually flipping coins. The professor returns, and has to guess which is random and which is simulated random.
        • Stock picking activity. Students guess whether each of six fictional stocks will rise or fall. The instructor picks if each stock will rise or fall by flipping a coin, and then asks the students if anyone got all six right. Typically, at least one student will, just by chance, even though it is clearly a matter of chance.
      • Homework Questions
        • Describe a case not discussed in class where someone (or many people) see signal where in fact there is only noise.
    • CLASS ELEMENTS  

      • Suggested Readings & Reading Questions
      • Clicker Questions
      • Discussion Questions
        • In the Stanley Kubrick film Dr Strangelove, which takes place during the Cold War, a rogue American general sends the code to send planes to drop nuclear bombs on Russia because he wants to start a nuclear war. One of the pilots believes that this could only have happened if Russia had struck first. Under these conditions (leaving aside the Doomsday Machine), what are the costs of a false positive (believing Russia struck first when it didn't) and what are the costs of a false positive (believing it is some kind of mistake, when in fact Russia did strike first)?
      • Class Exercises
      • Practice Problems
        • Suppose you have very bad migraines. Your doctors tell you migraines can be sparked by a lot of different things, which vary for different people: most kinds of food, coffee, stress, and/or bright sunlight. You want to find out what is causing your migraines, so you cut your diet to potatoes and then slowly add one food back at a time, waiting a few days between each addition to see if you get a migraine. The whole thing is stressful, and you’re so hungry and grumpy you lose your sunglasses on Monday. On Thursday, you eat chocolate again for the first time. That afternoon, you get a relatively small migraine. You are very sad, because chocolate is your very favorite food.
          • a.) Should you wait a few days and try eating a small amount of chocolate again, or should you ban it from your cupboards? Why?
          • b.)What if your migraines came with sudden, intense, lasting vertigo, and you work in skyscraper construction and can’t take off work?
      • Homework
        • Recount an instance in which you thought you made a right diagnosis of a situation, but then found out that you were wrong. Explain whether it was an instance of a false positive or a false negative. What makes it a false positive/negative as compared to a false negative/positive? Which type of error do you think would have been better to make in this situation, and why?
    • CLASS ELEMENTS

      • Introductory Examples
        • Saul's story of a physicist who cancelled a lecture five minutes in because the presenter wasn't sure how his error bars were calculated.
      • Suggested Readings & Reading Questions
      • Clicker Questions
        • If a President put forward a new policy proposal on health-care reform, which statement would make you feel more confident?
          • A. “The policy that I’m putting forward is the right policy for America. I guarantee that it is what’s best for the country.”
          • B. “I think that the policy I’m putting forward is the one that is most likely to be the right policy for America. There is no guarantee that it is will work -- in fact, I give it only a 75% chance, but the alternatives that have been presented are all much less likely to succeed.”
        • If you cross Oxford Street after class next Thursday, what is the likelihood that you will get hit by a car?
          • A. about 1 in 1,000 (one in a thousand)
          • B. about 1 in 100,000 (one in a hundred thousand)
          • C. about 1 in 10,000,000 (one in ten million)
          • D. about 1 in 1,000,000,000 (one in a billion)
          • E. about 1 in 100,000,000,000 (one in a hundred billion)
        • How large would the risk of getting hit by a car when crossing Oxford St. need to be in order to affect your plans?
          • A. about 1 in 1,000 (one in a thousand)
          • B. about 1 in 100,000 (one in a hundred thousand)
          • C. about 1 in 10,000,000 (one in ten million)
          • D. about 1 in 1,000,000,000 (one in a billion)
          • E. about 1 in 100,000,000,000 (one in a hundred billion)
      • Discussion Questions
        • Do you have any beliefs for which you have less than 1.0 credence but do not know how to do without?
        • Discuss any current controversial topic, but for each statement anyone makes, they have to give it a credence level.
      • Class Exercises
        • Students guess the answers to 10 binary questions, write credence levels for each one. See the answers, calculate your calibration.
        • Many examples given from recent science presentations.  
        • A arbitrary topic is chosen for small group discussion (e.g. “Does testing in the schools help or hurt education?”), but during the discussion the students have to state their credence level (by saying a number between 0 and 100%) after every statement that they make which could have a credence level associated with it.
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
      • Discussion Questions
        • What are the costs of underestimating one's credence level at a low level of confidence?
        • What are the costs of overestimating one's credence level at a high level of confidence?
        • What are some ordinary-life scenarios in which talk of confidence intervals might be useful? Hint: When deciding whether or not to stay in a job & wait for a raise, when financial planning, when betting, etc.
      • Class Exercises
        • Credence-calibration questionnaires to show students’ calibration, and training exercises.   

      • Homework Questions
        • Make ten predictions about ten events you expect to happen within the next week (total of 10, not 100!). Write your credence level for each one: that is, how confident you are it will happen within the next week, 1-100. After the week has passed, mark the ones that happened and calculate your calibration score. How well did you do? Did you predict some kinds of events better than others?
        • What’s a subject in which you have to make repeated predictions in everyday life? It could be academic, social, or personal. (Some possible examples: how you’ll do on tests, what grade you’ll get on papers, whether you will enjoy a new course, whether you’ll get jobs or grants you apply for, whether someone will respond to your message on a dating site, whether what you’re wearing will be appropriate for an event, etc.)
          • A.  How accurate are your predictions? What makes it difficult to be more accurate?
          • B. Is there any way that you could make your predictions more accurate? Why don’t you do this? (You may have a good reason; sometimes increased accuracy requires more time and effort than it’s worth.)
          • C.  How well calibrated is your confidence in your predictions? Do you tend to be overconfident or underconfident? Why?
          • D. Is there any way that you could make your confidence in your predictions better calibrated?
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
        • No readings
      • Homework
        • The purpose of today’s homework assignment is to give you some practice with the process of Fermi estimation which was outlined in the Santos reading. Make Fermi estimates on Question 1 and Question 2, below, following the steps: (i) identify what quantities to multiply together, (ii) make rough estimates of each of those quantities, (iii) do the math to get your answer, and (iv) state your answer clearly. Except for step iv, you do not need to write in complete sentences. Your answer can be formatted just like the sample answer to the sample question given below.
          • Question 1: How much money do you spend on coffee (or other warm beverage, if you are not a coffee drinker) in a year?
          • Question 2: If you took all of the household garbage in the US generated in a year and spread it out in the San Francisco Bay, what percentage of the surface of the bay would be covered in garbage?
          • Sample question: What is the total length of fingernail growth that one could achieve in a lifetime? Answer to sample question:
            • i) Quantities to multiply together: (length grown per clipping ) x ( of clippings per week) x ( of weeks per lifetime). ii) Estimates: length grown per clipping: .002 meters. of clippings per week: 1. of weeks per lifetime: 4160. iii) Do the math:(.002) x (1) x (4160) = 8.3 meters. iv) Answer: I estimate that a person’s fingernails grow about 8.3 meters (27 feet) in a lifetime. hw
      • Clicker Questions
        • How many pounds of food was thrown out (sent to landfills or incinerators) in the United States last year? 
          • A.Less than 100 million pounds
          • B.Between 100 million pounds and 1 billion pounds
          • C.Between 1 billion pounds and 10 billion pounds
          • D.Between 10 billion pounds and 100 billion pounds
          • E.More than 100 billion pounds 
        • Which of these three does the government spend the most on (including federal, state, and local government spending)?  The second most?
            1. Most on Education, then Incarceration, then Social Security
            1. Most on Education, then Social Security, then Incarceration  
            1. Most on Incarceration, then Education, then Social Security 
            1. Most on Incarceration, then Social Security, then Education 
            1. Most on Social Security, then Education, then Incarceration 
            1. Most on Social Security, then Incarceration, then Education 
      • Discussion Questions
      • Class Exercises
        • Together with the whole class, the professor shows how to develop Fermi-problem estimates of a given quantity, e.g. the amount per year that Americans spend on gas for personal transportation.  
        • In small groups, students work on several Fermi problems to develop facility with the approach.
        • In small groups, students use Fermi estimates to re-think the first-order, second-order, etc. parsing of US government spending on education, incarceration, and social security (following up the final activity from Topic XIIV, Orders of Understanding). The students frequently reach completely different orderings than they did in the previous class—and come within ~20% of the actual amounts spent.   
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • Suppose you flip a fair coin. Which of the following sequences of heads and tails is more likely: HHHHH or HTHHT?
          • A. HHHHH 
          • B. HTHHT 
          • C. They're equally likeley 
        • Dr. Six flips 6 coins at a time and counts how many heads and tails she gets. Every time she gets twice as many heads as tails (i.e., 4+heads), she eats an M&M. Dr. Twelve flips 12 coins at a time and counts how many heads and tails she gets. Every time she gets at least twice as many heads as tails (i.e., 8+heads), she eats an M&M. After 100 sets of flips, who will have eaten more M&Ms?
          • A. Dr. Six.  
          • B. Dr. Twelve. 
          • C. They’ll have eaten about the same number. 
        • Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.
          • A. Linda is a bankteller. 
          • B. Linda likes to cook and plays the trumpet. 
          • C. Linda is a writer. 
          • D. Linda is a bankteller and active in the feminist movement. 
        • A bat and a ball together cost $1.10. The bat costs a dollar more than the ball. How much does the ball cost?
          • A. Ten cents.  
          • B. Five cents.  
          • C. Other.  
      • Discussion Questions
        • Base rate neglect often turns up when people use statistics out-of-context to make their evidence seem stronger than it is. How might someone use base rate neglect to argue that some groups are more violent than others? How might you refute them (by pointing out that they are neglecting base rates)?
        • How is base rate neglect connected to the need for a control condition in RCTs?
      • Class Exercises
        • Small group exercises and clicker questions to demonstrate these effects with the students. 
      • Homework
        • Kahneman and Tversky (1974) end their paper on heuristics and biases with the words: “A better understanding of these heuristics and of the biases to which they lead could improve judgments and decisions in situations of uncertainty.” Give an example of a real-world situation in which one of the heuristics they discuss could bias judgments, and suggest a strategy for improving judgments. In other words, how might you get people to avoid using the heuristic as a basis for their judgment, and instead rely on a better alternative?
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • Scores on intelligence tests appear to be reliable (in that people reliably get similar measurements on repeated testing). Is IQ further a valid measurement, in that it measures what we want it to?  
          • A Yes 
          • B No 
          • C Don’t know/ other reaction 
        • The fact that you’ve found a way of reliably putting a number on individuals or groups doesn’t of itself imply that:
          • A. The number has any causal significance at all, or 
          • B. It has the kind of causal significance you take it to have. 
      • Discussion Questions
        • Why are our judgments about other people and groups of people so often mistaken? Hint: Allude to sources of both systematic and statistical error.
        • Which segments of society are most likely to be tempted by the Just World Fallacy? Why?
      • Class Exercises
      • Homework Questions
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
        • Case Studies (Possible Examples of Pathological Science):
          • Wolfe-Simon et al., 2011, "A bacterium that can grow by using arsenic instead of phosphorus."
          • Chaplin, 2007, "The Memory of Water."
          • Adam et al., "Measurement of the neutrino velocity with the OPERA detectorin the CNGS beam."
      • Clicker Questions
        • Identifying how many of Langumuir’s 6 Pathological Science indicators are in play in specific papers relating surprising scientice results.  
      • Discussion Questions
      • Class Exercises
        • Summaries of three relatively recent surprising science results (e.g. super-luminal neutrinos, bacteria with arsenic in their DNA, water with memory, cold fusion) and their follow-up in the scientific community are distributed among the groups. Each group explains the summary they read to other groups, so all have thought about each example. The groups discuss and vote with clickers on whether each article falls into the category of pathological science, poorly-done science, etc.   
      • Homework
        • Please read the article assigned to you based on your seating chart group (the seating chart for Week 9 is posted on the syllabus). These articles (from a variety of sciences) may be challenging to understand, but we encourage you to try to understand the main ideas from the articles and be prepared to discuss them in class on Wednesday. Please answer the following question for your homework: Based on Langmuir's criteria, do you think that the study conducted in the article you read would qualify as pathological science? In addition to a paragraph explanation summary of your thoughts, make sure to fill out and turn in the “Langmuir Scoresheet” as part of your assignment. [link]
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
          • Reading Question: Read the first two pages of the Fischer and Greitemeyer article. The reading discusses the selective-exposure effect, which is one instance of a broader phenomenon known as "confirmation bias." Provide an example of a selective exposure effect in real life (personal anecdote, news story, scientific study, etc.). Explain how the scenario you chose is an example of this type of bias, and how it might lead to suboptimal decision-making.
      • Clicker Questions
        • Cards have a letter on one side and a number on the other. Rule: If a card has a vowel on one side, it has an even number on the other side. Which card(s) would you turn: E, 7, 4, M 
      • Discussion Questions
        • On social media like Twitter and Facebook, people tend to follow their friends, especially friends that share their opinions. People also tend to share articles that support their viewpoint. This leads to selective exposure, wherein people are exposed much more to arguments that support what they already believe and much less to arguments against what they believe. In other words, there is a bias in what they see that confirms what they already think. How would this increase polarization of opinions?
        • Assuming you wanted to, how could you go about reducing your selective exposure?
        • What are the human motives that would sustain and encourage confirmation bias?
      • Class Exercises
      • Practice Problems
        • In Sweden, there is a syndrome called uppgivenhetssyndrom, in which children become completely comatose and unresponsive despite apparently having nothing physically wrong with them. Their reflexes and blood pressure remain normal.  Yet they are unresponsive to pain, and must be fed through feeding tubes stuck down their throats.  This syndrome exclusively affects refugee children in Sweden whose families are threatened with deportation; it has never been diagnosed outside of Sweden. It has affected hundreds of refugee children in Sweden, primarily children from former Soviet bloc states. Some children have remained comatose for years.  Initially, the families were deported anyway. However, photographs of unconscious children being deported on stretchers raised a public outcry. More recently, most families with an affected child have received reconsideration by the Board of Immigration. At present, the only known cure is for the family to be approved for permanent residency. Even after families are approved, it takes weeks or even months for the children to recover. Since the condition is thought to arise from external circumstances, doctors have primarily focused on keeping the children alive, not waking them up by medical means.
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
        • Students are presented historical graphs of improved published measurements of a physical parameter over the decades and must identify the ones that retrospectively show evidence of biases that could have been avoided by blind analysis.   
      • Discussion Questions
        • Why is it important for scientists to publicize results even if they don't get what they predicted?
      • Class Exercises
        • Students make a measurement which is somewhat tricky to perform with two-digit precision, and experimental conditions are set up to show that the part of the class that was “blinded” gets a more accurate result.  
        • There could be a variant of the above incorporating blind analysis into a more typical lab class, where there is an expectation about the results, half the class does the analysis blinded, half the class does the analysis unblinded.
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
      • Discussion Questions
        • Can you think of a group you've worked with that engaged in groupthink/herd thinking? What happened?
        • Each person think of the best-functioning group they've worked with. What made that group stand out?
      • Class Exercises
        • Students answer typical wisdom-of-crowd estimate questions using their clickers—but they can update their estimates as they see the histogram with the other students’ guesses. Afterwards, it is shown that the accuracy of the class’ mean estimate actually got worse as they continued to update their estimate—showing (if it works) that wisdom of crowd works best if the inputs are independent.    
      • Homework
        • One way to gather data about how people might respond to a product or idea is to conduct a “focus group,” in which several participants share their impressions in a group conversation. Based on the reading, why is or isn’t this a good way to obtain reliable information?
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
      • Discussion Questions
        • Think about the last time you were tempted to go do something fun with your friends when you had work to do. How did you cope with this value conflict between having fun with friends/developing friendships and getting your work done/learning/doing well in school? How do you usually cope with that conflict? Is there any way to fulfill both goals more completely? Will there always be a conflict between them (at times)?
        • Suppose you are the principal of a new charter school. You have a finite amount of money. Students and teachers want smaller classes. Teachers want better pay and benefits, and classroom equipment. Students want gym equipment, lockers, art and drama classes, and a track field. How do you go about prioritizing goals when your resources are limited and different stakeholders want different things?
      • Class Exercises
      • Homework Questions
        • List your top five values (for society).
        • What is one issue for which two or more of your top values might conflict? How do you cope with this value conflict? Do you have any beliefs about facts which allow you to avoid value conflict? If so, how sure are you that those beliefs are true?
        • Consider [X complicated decision relevant to class, in our case, the decisions for which students were about to design decision processes in small groups]. In 1-2 sentences, state why this decision is important. Then list 5 desiderata (desirable features) that you would like to be captured in a process for making this decision. Use the course topics for inspiration. For example, what desideratum could help one address systematic/statistical uncertainty in the decision making process?
    • CLASS ELEMENTS

      • Students work out a problem involving values and factual/scientific issues, using method used in Denver bullet study. 
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
        • Gene Drive briefing materials: [https://drive.google.com/file/d/1_EA4iR4N6XTR6yFbAY4MybDNfblAPF0w/view?usp=sharing]
      • Homework
        • Please write a question about the briefing materials (the assigned reading) that you would like to ask an expert. It should demonstrate that you've done the reading.
      • Clicker Questions
      • Discussion Questions
      • Class Exercises
        • In the two classes of this week, we try out an approach to random-sample-representative decision-making, using a panel of experts to answer questions generated by small deliberative groups, each with a moderator.   
    • CLASS ELEMENTS

      • Suggested Readings & Reading Questions
      • Clicker Questions
      • Discussion Questions
      • Class Exercises