Sometimes I come across psych journal articles that have been picked up in places like PsyPost. They often have headlines like: Emotional distress among voters tied to Trump’s populist appeal, research shows
We live in an age of -bait, and that means that it’s usually a good idea to dig beyond headlines if you see something that feels too good (or terrible) to be true.
Three common problems I’ve seen since the pandemic especially:
- Self-fulfilling design – observing variables that can only lead to the desired effect
- Preregistration headfakes – claiming that Variable X is important, and that you knew it all along, but not actually making that claim in your preregistration
- Zero adversarial hypotheses – a paper that can’t fathom plausible alternative explanations is a usually a paper to read *very* carefully
For examples of all of these, read on!
( The rest of this post is slightly edited from my original words on r/psychology )
The paper feels post-hoc-y (pronounced post-hokey) at best and full of confirmation bias at worst. Three things I noticed in reading the paper lead me in this direction:
1) The inclusion of Manichean outlook as part of the definition of populism is a self-fulfilling prophecy when it comes to negativity = populism
To begin, they didn’t *have* to include it:
While populism is a contested and multifaceted concept, a consensus has begun to emerge around the ideational approach to defining it. In this view, populism is comprised of three main tenets: (a) antielitism, (b) a Manichean outlook, and (c) people-centrism (Mudde, 2017).
And what is a “Manichean outlook”? Well…
Second, populism is typically a Manichean affair in that it divides society into two irreconcilable and antagonistic groups—the people and the elite—who are seen as forces for good and evil, respectively.
So… you’re including a literal black-or-white representation of ingroup vs. outgroup conflict in your *definition* of populism.
I see.
2) The pre-registration is sus
First, anytime someone says “we didn’t hypothesize X” in their paper,
We did not hypothesize a link between negative affect and people-centrism. In the empirical analysis, we found that higher levels of negative affect decrease people-centric attitudes.
check the preregistration. And, indeed, they only spoke of one real-ish H1:
2) What’s the main question being asked or hypothesis being tested in this study?
We hypothesize that negative emotions are associated with populist voting in real-stakes elections. Specifically, we test whether negative emotions (anxiety, anger, depression) inferred from Twitter are correlated with vote shares for the Leave campaign in the 2016 Brexit Referendum at the level of local authority districts.
Yet, in that very same preregistration, their lead-off point should raise your eyebrows:
1) Have any data been collected for this study already?
It’s complicated. We have already collected some data but explain in Question 8 why readers may consider this a valid pre-registration nevertheless.
The answer to Question 8:
8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
Data collection of Tweets is currently under way. However, none of the authors have looked at or run any analyses of the Tweets yet.
I see.
3) The relationship between people-centeredness and populism that complicates their story is underplayed:
Back to the article:
Although (link between negative affect and people-centrism) requires further theorizing and empirical research, the initial evidence suggests that populism is not simply synonymous with discontent.
Okay, good start…
The mix between (a) a largely negative outlook emphasizing crises and betrayal coupled with (b) a more hopeful belief in the power of the general will of the people and an optimism that radical change will improve voters’ well-being is, ultimately, what makes populism a set of ideas that go beyond just political grievance (cf. Curato, 2016; Hochschild, 2018; Montiel & Uyheng, 2020; Obradović et al., 2020; Reicher & Haslam, 2017)
Fair point! I’m with you so far!
An alternative explanation, worthy of further research, is that negative affect is likely to raise demand for strong leaders as opposed to more inclusive, people-centered approaches to politics. This is in line with the focus of some populism scholars on the role of strong or personalistic leadership, particularly in the face of (perceived) threats or crises (Moffitt, 2016).
Okay, so, back to the tautology? Not a fan. And then:
Alternatively, it may be the case that negative affect increases biases and ingroup preferences, in which case the extent to which affect will be related to people-centrism is likely to hinge greatly on who is counted among “the people” (see, e.g., Banks, 2016). Further research where survey questions might (preferably experimentally) vary this aspect could be a fruitful avenue for future investigation.
BRUH. If the negativity (Manicheanism) is baked into your methods, you know you’re going to find that exact correlation!
Just because you can make the correlation appear doesn’t mean it’s a *meaningful relationship*. I know that murder rates and ice cream consumption strongly correlate – especially if they’re the only thing in my model!
PS: That’s not science, that’s selective analysis.
BONUS THOUGHT:
For people who are thinking “Well what’s a good H2 / HAlt then?”
I’m not an expert in the field, but I’d say that the people-centeredness might correlate with many positive things, especially if the populism is oriented toward seeking justice for the aggrieved. (IDK correlate it with belief in a just world and see what you get)
OR!
What if the Manichean outlook – people centeredness relationship was mediated by something we know has really interesting political implications: Truth in Reconciliation?
For instance, what about a populism that embraced Robert Reich’s explanations on how neoliberal economic policies (pushed by Democrats and Republicans) screwed over a lot of Americans?
I would feel better about the paper if the authors had included a hypothesis antagonistic to their findings, but alas:
Nevertheless, although the data suggest that populism may have positive emotional aspects to it, it remains dominated by its negative antielitist and Manichean components when it comes to consequential behavior at the polls, given that negative affect ultimately strongly predicts populist voting and election results.
So… don’t look behind the curtain. Got it, Oz.
TL;DR:
This paper is a tautology, the preregistration is suspect, and inconvenient results are not sufficiently engaged.
C-, rewrite and resubmit.
Intersection 9 