Social science and cultural narratives: Truth over ideology
For the past several years, social scientists have been increasingly aware of what is called the “replication crisis” in their disciplines.
In 2015, the Reproducibility Project at the University of Virginia tried to replicate the results of nearly 100 studies published in psychology journals. Even after going “through extensive measures to remain true to the original studies,” they could only replicate the original findings in 36 percent of the studies, and even then, the effects “were smaller than the initial studies effects.”
This problem with replicating results isn’t limited to psychology or the social sciences. As Pascal-Emmanuel Gobry of the Ethics and Public Policy Center has noted, medicine is often afflicted with a similar replicability problem.
As the saying goes, the first step in fixing a problem is acknowledging you have one. However, given the personal, professional, and political stakes often at work here, that’s easier said than done.
That’s why a recent bit of honesty by a publication you may never have heard of is so welcome.
Back in 2014, the journal Social Science & Medicine published a study entitled “Structural stigma and all-cause mortality in sexual minority populations.” The study attempted to measure the impact of anti-gay prejudice on the death rate of gay men and lesbians.
The study concluded that “Sexual minorities living in communities with high levels of anti-gay prejudice experienced a higher hazard of mortality than those living in low-prejudice communities.” Specifically, this increased “hazard of mortality” “[translated] into a shorter life expectancy of approximately 12 years . . . for sexual minorities living in high-prejudice communities.”
The study was used as scientific confirmation of an emerging cultural narrative. Not only is anti-gay prejudice wrong, we are often told, it can even be lethal.
Well, not so fast. Mark Regnerus of the University of Texas did what social scientists in the peer-review process are supposed to do… He set out to replicate the results of that study.
Going over the same data set, he found multiple errors, flawed assumptions, and possible “confounding variables,” possible factors that the researchers failed to take into account: ethnicity, age, sex, and smoking, to name but four.
In a 2017 study published in the same journal, Regnerus, who by the way is a Christian, concluded “Attempts to replicate the study . . . repeatedly failed to generate the original study’s key finding on structural stigma.”
To put it mildly, questioning the findings didn’t go over so well. Nathaniel Frank of the What We Know project, which is a gay research portal, summed up the views of many who commented when he wrote that, “Mark Regnerus destroyed his scholarly credibility when . . . he allowed his ideological beliefs to drive his conclusions,” and that “there’s an enormous body of research showing the harms of [sexual] minority stress, and Regnerus is simply not a trustworthy critic of this research.”
Except, as it turns out, Regnerus is trustworthy. Earlier this month Social Science & Medicine announced that the original study had been retracted at the original authors’ request. In response to Regnerus’ findings, they hired another group to try and replicate their original findings, and that group came to the same conclusion as Regnerus.
As one social science website put it, “It was a mistake of the journal Social Science & Medicine to publish that original paper, but, on the plus side, they published Regnerus’s criticism. Much better to be open with criticism than to play defense.”
Exactly. Mistakes like this are inevitable. All we can ask is that, once the mistake has been pointed out, ideology should not be allowed to get in the way of the truth.
The journal did the right thing, which is as welcome as it is rare.
Originally posted at breakpoint.org
