Tag: the experimenter expectancy effect

  • Science Corrects Itself: The Embodied Cognition Story 

    Science Corrects Itself: The Embodied Cognition Story 

    Try giving someone directions while sitting on your hands. It is harder than it should be. Your hands start drawing the map, pointing at the turns, shaping the roundabout — even on the phone, where nobody can see a thing. The body gets involved in work that supposedly belongs to the brain, and for the past twenty-five years psychology has had a name for the idea that this is no accident: embodied cognition. On the podcast this week I walk through the whole arc of this idea — the exciting findings, the collapse of many of them, and the genuinely hopeful way the field responded. This post covers the highlights, with sources at the bottom for students and instructors.

    What Embodied Cognition Claims

    The picture of the mind most of us inherited treats the brain as a computer: input, processing, output, with the body serving as the peripherals. Embodied cognition pushes back. It says thinking is built out of the body’s experiences — that concepts are grounded in sensing and moving. The linguist George Lakoff and the philosopher Mark Johnson laid the intellectual groundwork in their 1980 book Metaphors We Live By, pointing out how much abstract thought leans on physical metaphor. A warm person. A dirty trick. You grasp an idea, weigh your options, carry a heavy secret. Their claim was that these are not decorations on top of thought. They are the structure of thought, learned through the body.

    The Findings That Made Everyone Believers

    The strongest evidence came from neuroscience. In 2004, Olaf Hauk, Ingrid Johnsrude, and Friedemann Pulvermüller put people in an fMRI scanner and had them passively read action words — lick, pick, kick. Your motor cortex is organized like a map of the body, and reading each word activated motor areas overlapping or adjacent to the regions that control the tongue, the fingers, and the feet. Nobody moved. The meaning of the word reached the body’s control panel anyway. This finding, published in Neuron, has been repeated and extended many times, and it still stands.

    Behavioral evidence followed. Arthur Glenberg and Michael Kaschak (2002) reported the action-sentence compatibility effect: people judging sentences like “Art handed you the pen” responded faster when their hand movement matched the direction of the action in the sentence, suggesting that understanding language involves quietly simulating it with the motor system.

    Then the idea got glamorous. John Bargh, Mark Chen, and Lara Burrows (1996) reported that students who unscrambled sentences containing elderly-stereotype words — Florida, bingo, gray — walked more slowly down the hallway afterward, timed by an experimenter with a stopwatch. (Strictly speaking this is social priming, with the arrow running from concept to body rather than body to concept, but the two literatures grew up together and, as you will see, fell together.) Williams and Bargh (2008) found that briefly holding a hot coffee led people to rate a stranger’s personality as warmer. Zhong and Liljenquist (2006) reported the “Macbeth effect”: copying a story about an unethical act made cleansing products seem more desirable. Strack, Martin, and Stepper (1988) found that holding a pen in the teeth — which forces a smile — made cartoons seem funnier. And Carney, Cuddy, and Yap (2010) reported that two minutes of expansive “power posing” raised testosterone, lowered cortisol, and increased risk taking in a sample of 42 people. Amy Cuddy’s 2012 TED talk on that finding became one of the most watched ever.

    I Believed It Too

    Full disclosure: in my book Psych Experiments, I suggested that readers could try the hot coffee study for themselves. It was published in Science. Textbooks taught it. I taught it. This next section is therefore a little personal.

    Then the Replications Came

    Stéphane Doyen and colleagues (2012) redid the elderly walking study with infrared sensors instead of a stopwatch and a larger sample: no effect. Their second experiment is the one every student should know. They manipulated what the experimenters expected — and a walking effect appeared only when the experimenters believed participants would walk slowly. The effect appeared to live in the experimenters, a modern Clever Hans story and a clear demonstration of why double-blind procedures exist.

    Dermot Lynott and colleagues (2014) ran three high-powered direct replications of the warm-pack version of the coffee study — 861 participants in total. No effect in any lab. Brian Earp and colleagues (2014) obtained the original Macbeth effect materials, ran samples about five times larger than the original across multiple countries, and found nothing. A seventeen-lab registered replication led by E.-J. Wagenmakers (2016) tested the pen-in-the-teeth study: the original difference of 0.82 points shrank to 0.03 — a flat line. Eva Ranehill and colleagues (2015) replicated power posing with roughly five times the participants: people felt more powerful, but the hormone and behavior effects vanished, and a p-curve analysis by Simmons and Simonsohn (2017) suggested the supporting literature looked like noise. In 2016, Dana Carney — first author of the original power posing paper — publicly stated that she no longer believed the effect was real.

    And in 2022 came the one that stings: an eighteen-lab, pre-registered replication of the action-sentence compatibility effect, the flagship behavioral finding of the whole field. Not one lab found it, and the pooled estimate of the effect was essentially zero. Look at the author list of that paper: Kaschak and Glenberg — the original discoverers — helped run it and signed it.

    Why Did the Original Studies Find Effects?

    Not fraud, for the most part. Something more ordinary and more instructive. Tiny samples (30 to 42 people) produce results that swing widely from sample to sample, and some of those swings are large by pure sampling luck. Publication bias means journals printed the lucky swings while null results went in the file drawer. Researcher degrees of freedom — which measure to report, which participants to exclude, when to stop collecting data — offered many innocent-looking forks in the road, and enough forks will usually lead to a publishable result. And experimenter expectancy effects, as Doyen demonstrated directly, can manufacture a finding when the people running the study know the hypothesis and interact with participants.

    The Rebuild: Adversarial Collaboration

    Failed replications sparked arguments, and some objections from original authors were reasonable — Strack noted, for example, that the replication labs used visible video cameras, which may make participants self-conscious. The field’s answer to this stalemate is the best part of the story. In 2022, the Many Smiles Collaboration published, in Nature Human Behaviour, a pre-registered test of the facial feedback hypothesis designed jointly by proponents and critics — Strack himself was on the team — with 3,878 participants in 19 countries. The verdict landed in the middle: deliberately posing or mimicking a smile does nudge felt happiness, a real but modest effect, while the pen-in-the-mouth method came back inconclusive. That matches a 2019 meta-analysis by Coles and colleagues: facial feedback is real, and it is small.

    What’s Left Standing

    Here is where things honestly stand: the neuroscience core of embodied cognition is in decent shape — sensory and motor systems really do participate when we process meaning. The flashy social-priming findings have collapsed: coffee cups, cleansing products, and hormone-shifting poses did not survive rigorous testing, and the flagship behavioral effect could not be found by the very people who discovered it. The open question is whether the body’s involvement is necessary for understanding or is more of an echo. Meanwhile the field rebuilt its methods — pre-registration, multi-lab studies, samples in the thousands — and the original researchers helped run the studies that overturned their own findings. People call this the replication crisis. I’d call it science doing exactly what it is supposed to do. And the next edition of Psych Experiments will treat the hot coffee study accordingly: run it, and expect to find nothing — because finding nothing, when the original claim was world famous, is a real result.

    Psychology Terms in This Article

    Embodied cognition — The theory that thinking is grounded in the body’s sensory and motor systems rather than consisting only of abstract symbol processing. In this episode, it is the framework behind everything from the kick-lick-pick fMRI study to power posing.

    Replication — Repeating a study to see whether its results hold up; a direct replication reproduces the original methods as closely as possible. The seventeen-lab test of the pen-in-the-teeth study and the eighteen-lab test of the action-sentence compatibility effect are direct replications.

    Statistical power — A study’s ability to detect an effect if the effect is real, which depends heavily on sample size. The original power posing study had 42 participants; its replication had roughly five times as many, which is why its null result carries weight.

    Publication bias — The tendency for studies that find effects to get published while null results stay in the file drawer, so the published record shows only the successes. This is one reason so many small early studies pointed the same direction.

    Demand characteristics — Cues in a study that hint to participants what the researcher expects, nudging their behavior. Visible video cameras in the smile replications were debated as a possible source.

    Experimenter expectancy effect — The experimenter’s own beliefs leaking into how a study is run and measured, as when a walking-speed effect appeared only for experimenters who expected it. Clever Hans, the “counting” horse, is the classic historical example.

    Pre-registration — Publicly recording a study’s hypotheses, methods, and analysis plan before collecting data, which closes off after-the-fact flexibility. Every major replication discussed in this episode was pre-registered.

    Adversarial collaboration — A study designed and run jointly by researchers who disagree, with the rules agreed in advance. The Many Smiles Collaboration, which included the original smile-study author alongside his critics, is the model case.

    References

    Bargh, J. A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 71(2), 230–244. https://doi.org/10.1037/0022-3514.71.2.230

    Carney, D. R., Cuddy, A. J. C., & Yap, A. J. (2010). Power posing: Brief nonverbal displays affect neuroendocrine levels and risk tolerance. Psychological Science, 21(10), 1363–1368. https://doi.org/10.1177/0956797610383437

    Coles, N. A., Larsen, J. T., & Lench, H. C. (2019). A meta-analysis of the facial feedback literature: Effects of facial feedback on emotional experience are small and variable. Psychological Bulletin, 145(6), 610–651. https://doi.org/10.1037/bul0000194

    Coles, N. A., March, D. S., Marmolejo-Ramos, F., et al. (2022). A multi-lab test of the facial feedback hypothesis by the Many Smiles Collaboration. Nature Human Behaviour, 6(12), 1731–1742. https://doi.org/10.1038/s41562-022-01458-9

    Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012). Behavioral priming: It’s all in the mind, but whose mind? PLoS ONE, 7(1), e29081. https://doi.org/10.1371/journal.pone.0029081

    Earp, B. D., Everett, J. A. C., Madva, E. N., & Hamlin, J. K. (2014). Out, damned spot: Can the “Macbeth Effect” be replicated? Basic and Applied Social Psychology, 36(1), 91–98. https://doi.org/10.1080/01973533.2013.856792

    Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565. https://doi.org/10.3758/BF03196313

    Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004). Somatotopic representation of action words in human motor and premotor cortex. Neuron, 41(2), 301–307. https://doi.org/10.1016/S0896-6273(03)00838-9

    Lakoff, G., & Johnson, M. (1980). Metaphors We Live By. University of Chicago Press.

    Lynott, D., Corker, K. S., Wortman, J., Connell, L., Donnellan, M. B., Lucas, R. E., & O’Brien, K. (2014). Replication of “Experiencing physical warmth promotes interpersonal warmth” by Williams and Bargh (2008). Social Psychology, 45(3), 216–222. https://doi.org/10.1027/1864-9335/a000187

    Morey, R. D., Kaschak, M. P., Díez-Álamo, A. M., Glenberg, A. M., Zwaan, R. A., Lakens, D., et al. (2022). A pre-registered, multi-lab non-replication of the action-sentence compatibility effect (ACE). Psychonomic Bulletin & Review, 29, 613–626. https://doi.org/10.3758/s13423-021-01927-8

    Ranehill, E., Dreber, A., Johannesson, M., Leiberg, S., Sul, S., & Weber, R. A. (2015). Assessing the robustness of power posing: No effect on hormones and risk tolerance in a large sample of men and women. Psychological Science, 26(5), 653–656. https://doi.org/10.1177/0956797614553946

    Simmons, J. P., & Simonsohn, U. (2017). Power posing: P-curving the evidence. Psychological Science, 28(5), 687–693. https://doi.org/10.1177/0956797616658563

    Strack, F., Martin, L. L., & Stepper, S. (1988). Inhibiting and facilitating conditions of the human smile: A nonobtrusive test of the facial feedback hypothesis. Journal of Personality and Social Psychology, 54(5), 768–777. https://doi.org/10.1037/0022-3514.54.5.768

    Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., et al. (2016). Registered Replication Report: Strack, Martin, & Stepper (1988). Perspectives on Psychological Science, 11(6), 917–928. https://doi.org/10.1177/1745691616674458

    Williams, L. E., & Bargh, J. A. (2008). Experiencing physical warmth promotes interpersonal warmth. Science, 322(5901), 606–607. https://doi.org/10.1126/science.1162548

    Zhong, C.-B., & Liljenquist, K. (2006). Washing away your sins: Threatened morality and physical cleansing. Science, 313(5792), 1451–1452. https://doi.org/10.1126/science.1130726