Is It Offensive To Declare A Social Psychological Claim Or Conclusion Wrong?

By Lee Jussim

Science is about “getting it right” – this is so obvious that it should go without saying. However, there are many obstacles to doing so, some relatively benign (an honestly conducted study produces a quirky result), others less so (p-hacking). Over the last few years, the focus on practices that lead us astray have focused primarily on issues of statistics, methods, and replication.

These are all justifiably important, but here I raise the possibility that other, more subjective factors, distort social and personality psychology in ways at least as problematic. Elsewhere, I have reviewed what I now call questionable interpretive practices – how cherrypicking, double standards, blind spots, and embedding political values in research all lead to distorted conclusions (Duarte et al, 2014; Jussim et al, in press a,b).

But there are other interpretations problems. Ever notice how very few social psychological theories are refuted or overturned?   Disconfirming theories and hypotheses (including the subset of disconfirmation, failures to replicate) should be a normal part of the advance of scientific knowledge. It is ok for you (or me, or Dr. I. V. Famous) to have reached or promoted a wrong conclusion.

In social psychology, this rarely happens. Why not? Many social psychologists seem to balk at declaring some claims “wrong.” This seems to occur primarily for three reasons. The first is that junior scholars, especially pre-tenure, may justifiably feel that potentially angering senior colleagues (who may later be called on to write letters for promotion) is not a wise move. That is the nature of the tenure beast, but it only explains the behavior of, at most, a minority. What about the rest of us?

The second reason is essentially social (i.e., not scientific). Declaring some scientific claim to be “wrong” is, I suspect, often perceived as a personal attack on the claimant. This probably occurs because it is impossible to declare some claim wrong without citing some article making the claim. Articles have authors, so that declaring a claim wrong is tantamount to saying “Dr. Earnest’s claims are wrong.” This problem is further exacerbated by the fact that theories, hypotheses, and phenomenon often become identified with either the originators or apostles (prestigious researchers who popularize them). Priming social behavior? Fundamental attribution error? Bystander effect? System justification? Implicit racism?  There are individual social psychologists associated with each of these ideas. To challenge the validity, or even the power or generality of such ideas/effects/theories/hypotheses risks being interpreted as something more than a mere scientific endeavor – it risks being seen as a personal insult to the person identified with them. Thus, declaring a claim “wrong” risks being seen, not as a scientific act of theory or hypothesis disconfirmation, but as a personal attack — and no one supports personal attacks.

The third reason is grounded in a very unique philosophy of science perspective – namely, that almost every claim is true under some conditions (for explicitly articulated versions of this, see Greenwald, Pratkanis, Leippe, & Baumgardner, 1986; McGuire, 1973, 1983). As such, we have a great deal of research on “conditions under which” some theory or hypothesis holds, but very little research providing wholesale refutation of a theory or hypothesis. I have heard apocryphal stories of prestigious researchers declaring (behind closed doors) that they only run studies to prove what they already know and that they can craft a study to confirm any hypothesis they choose. These apocrypha are not evidence – but the evidence of p-hacking in social psychology and elsewhere (e.g., Ioannidis, 2005; Simmons et al, 2012; Vul et al, 2009) raises the possibility that some unknown number of social psychologists conduct their research in a manner consistent both with these apocrypha and with the notion that everything is true under some conditions. If every claim is true under some conditions, then massive flexibility in methods and data analysis in the service of demonstrating almost any notion becomes, not a flaw to be rooted out of science, but evidence of the “skill” and “craftsmanship” of researchers, and of the “quality” of their research. In this context, declaring any scientific claim, conclusion, hypothesis or theory “wrong” becomes unjustified. It reflects little more than ignorance of this “sophisticated” view of science, and arrogance in the sense that no one, according to this view, can declare anything “wrong” because it is true under some conditions. As such, declaring some claim wrong can again be viewed as an offensive act.

The idea that claims cannot be “wrong” because “every claim is true under some circumstances” goes too far for two reasons. First, some claims are outright false, such as “the Sun revolves around the Earth.” Furthermore, even if two competing claims are both correct under some conditions, this does not mean they are equally true. Knowing that something is true 90% of the time is quite different than knowing it is true 10% of the time. Claiming that some phenomena is “powerful” or “pervasive,” when the data show it is only rarely true, is wrong. Let’s say that, on average, stereotype biases in person perception are not very powerful or pervasive – which they are not (Jussim, 2012 – multiple meta-analyses yield an average estimate of r = .10 for such biases). Isn’t it better to point out that the field’s long history of declaring them to be powerful and pervasive is wrong (at least when the criterion is the field’s own data), than to just report the data without acknowledging its bearing on longstanding conclusions?

This reluctance to declare certain theories or hypotheses wrong risks leading social psychology to become populated with a plethora of “… undead theories that are ideologically popular but have little basis in fact” (Ferguson & Heene, 2012, p. 555). This amusing phrasing cannot be easily dismissed – ask yourself, “Which theories in social psychology have ever been disconfirmed?” Indeed, a former President of the National Research Council, Dr. Bruce Alberts and editor of science put it this way (quoted in The Economist, 2013):

“And scientists themselves need to develop a value system where simply moving on from one’s mistakes without publicly acknowledging them severely damages, rather than protects, a scientific reputation.’”

I agree. It is ok to be wrong. In fact, if one engages in enough scientific research for a long enough period of time, one is almost guaranteed to be wrong about something. Good research at its best can be viewed as systematic, creative, and informed trial and error. But that includes … error! Both being wrong sometimes, and correcting wrong claims are integral parts of healthy scientific processes.

Furthermore, from a prescriptive standpoint of how science should proceed, I concur with Popper’s (1959/1968) notion that we should seek to disconfirm theories and hypotheses. Ideas left standing in the face of strong attempts at disconfirmation are those most likely to be robust and valid. Thus, rather than being something we social psychologists should shrink away from, bluntly identifying which theories and hypotheses do not (and do!) hold up to tests of logic and existing data should be a core component of how we conduct our science.

 

References

Duarte, J. L., Crawford, J. T., Stern, C., Haidt, J., Jussim, L., & Tetlock, P. E. (2014). Political diversity will improve social psychological science. Manuscript that I hope is on the verge of being accepted for publication.

The Economist (October 19, 2013). Trouble at the lab. Retrieved on 7/8/14 from:

http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble

Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychology’s aversion to the null. Psychological Science, 7, 555-561.

Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress? Psychological Review, 93, 216-229.

Ioannidis, J. P. A. (2005).  Why most published research findings are false. PLOS Medicine, 2, 696-701.

Jussim, L. (2012). Social perception and social reality: Why accuracy dominates bias and self-fulfilling prophecy. NY: Oxford University Press.

Jussim, L., Crawford, J. T., Anglin, S. M., & Stevens, S. T. (In press a). The politics of social psychological science II: Distortions in the social psychology of liberalism and conservatism. To appear in J. Forgas, K. Fiedler, & W. Crano (Eds.), Sydney Symposium on Social Psychology and Politics.

Jussim, L. Crawford, J. T., Stevens, S. T., & Anglin, S. M. (In press b). The politics of social psychological science I: Distortions in the social psychology of intergroup relations. To appear in P. Valdesolo & J. Graham (Eds.), Bridging Ideological Divides: Claremont Symposium on Applied Social Psychology.

McGuire, W. J. (1973). The yin and yang of progress in social psychology: Seven koan. Journal of Personality and Social Psychology, 26, 446-456.

McGuire, W. J. (1983). A contextualist theory of knowledge: Its implications for innovation reform in psychological research. Advances in Experimental Social Psychology, 16, 1-47.

Popper, K. R. (1959/1968). The logic of scientific discovery. New York: Harper & Row.

Simmons, J.P., Nelson, L.D., & Simonsohn, U. (2011).  False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366.

Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009). Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4, 274-290.