8. Why Civility Matters in the Replicability Discourse

I have to admit: I am more than a bit nervous to write this post. Many “big” events have transpired within the last week of the ongoing discourse regarding replicability in psychological science, and the resulting exchanges on Twitter, Facebook, and the blogosphere have seemed incredibly heated and personal. Most centrally, the widely-discussed (and embargo-leaked) commentary critiquing the Reproducibility Project was released in Science:

Only to be rapidly responded to with–err, preceded by–thoughtful critiques of the critique (and even critiques of the critiques of the critique) by the Reproducibility Project, and many others:

In the midst of all of the online activity, I have seen some pretty ugly behaviour, from one-off snide comments, to elaborate flame wars, including (but not limited to): name-calling, mean-spirited poetry, attempts to shame individuals from participating in the discourse, and appeals to authority intending to silence critical discussion. This is not the level of “scientific”communication that our discipline deserves–least of all now. And I felt like I needed to blog about this recent trend in online communication about replicability, because I actually feel hurt for many of the individuals involved, many of whom I hardly know at all.

For me, Simine Vazire comes close to hitting the nail on the head with this tweet on how psychologists should be conducting themselves in the public replicability discourse:

However, I would go one step further, and remove many of the qualifications; these rules shouldn’t just apply to those who are conducting replications, failed, successful or anything in between. Instead, these rules could and should also apply to anyone wanting to partake in the replicability discourse, no matter the side. Indeed, there has been a lot of bad communication behaviour to go around on both sides of the replicability aisle.* Dan Gilbert, for example, previously occupied the position of offering mean-spirited commentary when referring to those interested in replicability as “second-stringers”, and now finds himself the target of a considerable amount of online vitriol. I lament both sides of his exemplar.

So why, exactly, is civility so important to public scientific discourses, like issue of replicability ?

  1. Civility is inviting; mean-spiritedness is repelling. My interpretation of the purpose for the #OpenScience movement was to get more/all scientists onboard with becoming more transparent about their data, methods, materials, and analyses. Call me a closeted self-determination theorist, but I believe that the surest way of accomplishing this goal is for the #OpenScience community to be warm, inviting, and encouraging of people making changes to the way they conduct their science–not to bludgeon them into it with name-calling or professional threats/guilt-trips.Perhaps some feel as though offering mean-spirited comments targeting prominent people in the field is the only way to get these prominent people to change their scientific practices to become more Open Science friendly .** But even if that was for sure the case–and I am skeptical of the efficacy of this strategy–such a strategy can have chilling effects on third-parties who are impacted by the replicability discourse, namely, junior colleagues. An unnamed junior colleague of mine, for example, recently disclosed to me:
    …right now I have a study I didn’t cook at all that has N = 225 and p = .048, and I don’t want to publish it cause if someone took their [replicability] ray gun at me, I would look suspicious.

    This kind of fear hurts both the field–which misses out on this research–and my junior colleague–who can scarcely afford to avoid publication attempts, irrespective of how “borderline” their effects are. That my junior colleague insisted on remaining anonymous for their protection (because they are that scared of being called a “hack”) only serves to further highlight the repelling fear of mean-spiritedness.

  2. Civility leads to focus on the actual issues; mean-spiritedness distracts from the actual issues. Sanjay Srivastava aptly characterizes the situation:

    In other words, when we are not civil, our audiences will focus their reactions on our mean-spiritedness–not the substance of our arguments. Sometimes these reactions may happen honestly/automatically, whereas other times they may be a more deliberate attempt to create a straw-man counter-argument. Either way, the end-result is the same (so let’s not waste time trying to distinguish cases on this basis): less civility = less engaged discussion on the arguments that matter. For example, implying that a critic–be them “pro-replicability” or “replicability-skeptic”–of our position is a hack (on whatever subjective/quantitative basis), and therefore their critique shouldn’t be attended to, doesn’t actually address the substance of their critique. And I’ll espouse the bold position that I think few people in our discipline are that self-serving/stupid/mindlessly guild-promoting/evil, and therefore don’t deserve to have their voices silenced in this dismissive way. 

  3. Civility is a positive experience for both parties; mean-spiritedness is a negative experience for at least one party. I have had some riveting debates regarding replicability-related issues with many of my colleagues, and some of the most memorable happened with instructors and colleagues with whom I disagreed the most about replicability-matters. But the funny thing about these disagreements is that they were also incredibly enjoyable–arguably more enjoyable than discussions with colleagues who shared my beliefs– in large part because my counterparts and I argued passionately, but respectfully. I’ve also noticed that neutral or undecided third-parties enjoy such debates more when they are conducted with civility. Mean-spiritedness, alternatively, guarantees to leave at least one party (if not all) will leave feeling crappy. And there are already more than enough negative-affect-inducing experiences in our profession: our papers and grants are frequently rejected; we are always under the gun of some professional hurdle, be it passing defenses/getting jobs/getting tenure/etc.,; and we receive constant critical commentary about our research with massive delays until our personal rewards are realized. In other words, academic life is enough of a struggle already, without having to deal with colleagues being mean-spirited to one another in public domains.

Despite my handwringing about civility, I actually do think there is a time and/or place for mean-spiritedness. Believe it or not, I get the impulse to run someone down, or make a snappy and disparaging remark about a paper/blog-post/tweet, etc., especially when someone runs amok of my scientific/political values.*** Such responses help us to blow off much-needed steam, generate clicks for our blogs, and help us to feel closer to like-minded individuals. But at the end of the day, civility is a surer path than mean-spiritedness, for our goal of bringing others into the family of open and replicable science. So if you come across an article/story/blog-post/tweet that gets your goat, save the funny, but mean-spirited commentary for your private exchanges with like-minded friends. A civil approach to the public discourse about replicability, instead, will facilitate invitation and inclusion, a focus on the actual issues, and a more positive experience for all–exactly the type of discourse our science and its members deserve.

Is It True? Is It Necessary? Is It Kind?

Oh! Stay, dear child, one moment stay,

Before a word you speak,

That can do harm in any way

To the poor, or to the weak;

And never say of any one

What you’d not have said of you,

Ere you ask yourself the question,

“Is the accusation true?”

And if ’tis true, for I suppose

You would not tell a lie;

Before the failings you expose

Of friend or enemy:

Yet even then be careful, very;

Pause and your words well weigh,

And ask it it be necessary,

What you’re about to say.

And should it necessary be,

At least you deem it so,

Yet speak not unadvisedly

Of friend or even foe,

Till in your secret soul you seek

For some excuse to find;

And ere the thoughtless word you speak,

Ask yourself, “Is it kind?”

When you have ask’d these questions three—


Ask’d them in all sincerity,

I think that you will find,

It is not hardship to obey

The command of our Blessed Lord,—

No ill of any man to say;

No, not a single word.

*Full disclosure: I identify as a “concerned citizen-scientist”, who believes the replication “crisis” in psychology is real, and that there is much we can do to begin improving the quality of the science we produce. I have even written a couple of articles about possible solutions for psychology, am attempting to bring the “good-word” of replicability concerns to other science communities that I am a part of, and am trying to make Open Science practices an increasingly large part of my  own research. And though I feel like I shouldn’t have to disclose all of this, in order to be taken seriously, I anticipate that I might need to…

**I have also seem some commentary to the effect that mean-spiritedness is justified because of power disparities between those who are espousing and using Open Science practices, and those who are not. Without getting into the matter in more detail, I think in this particular case, this argument can be a specious barrier offered by some to avoid challenging themselves to a higher standard of civil discourse. Many among the vocal “pro-replicability”crowd, for example, are men**** occupying tenure-track positions at R1 or R1-ish schools. And many of the vocal “replicability skeptics” crowd are also men occupying tenure-track positions at R1 or R1-ish schools. Might there be a relative prestige/power differential here? Perhaps. But by any absolute standard, members of both groups occupy highly privileged positions.

***Trust me, I have a pretty strong pun/haiku/limerick game when it comes to the stuff I disagree with…

****This observation is not meant to overlook the vocal female contributors to the discourse, many of whom (e.g., Samine Vazire, Liz Page-Gould, Erika Salomon, etc.,) who, in my opinion, are some of the most consistent sources of wisdom within the replicability discourse. Rather, much of the mean-spiritedness I have observed has been carried out by men. So when I see appeals to power differentials to justify this conduct, well… (see **)

[I am cyynically awaiting my r-index/incredibility index/p-curve]

3 thoughts on “8. Why Civility Matters in the Replicability Discourse

  1. It seems to me that it’s possible for a genuine power imbalance to be a factor here, beyond the gender issue. The defenders of the status quo are, from what I can see (and with exceptions, of course), skewed towards being older, fully tenured, in some cases AEs of prestigious journals, and having their own profitable/high-status sidelines. The pro-replicators are generally (again, with exceptions), younger, often untenured regardless of the track they’re on, and still at the stage where they have to actually write good stuff in their articles to get past the reviewers. (There are women on both sides of the discussion, too: I haven’t seen whether Lisa Feldman Barrett has written anything defending the status quo in the past week, but I seem to recall that she defended it quite strongly in an op-ed somewhere when the RP:P article first came out.)

    If any situation where you believe that the other side is not engaging in sincere dialogue, polite tone is inevitably going to suffer. That’s just human (oh, if only someone would start a scientific discipline that could teach us about human behaviour). It seems that many in the pro-replication camp are not convinced that the objections of the “there’s no problem” camp are entirely based in objective science, without any consideration of, say, the impact that a generally increased level of public skepticism of social psychology might bring for their own interests. It may also be (but I’m less able to discern, being in the other camp) that the status quo people perceive the replicators as making unwarranted attacks on their science, based on jealousy or something. In any case, we’re not talking about a sincere, purely scientific dispute over the true atomic weight of carbon here.

    Civility is a cherry on the top; you can have it when everything else is being done correctly. When a hedge fund buys up my neighbourhood (and my city council representative’s vote), and a few weeks later someone comes round to my house to tell me that I have to leave so they can demolish it to make way for this new luxury condo they’re building, I don’t really care that they say “Please” and “Thank You”, and I’m not going to use that language with them. To return to your example of gender-based power imbalances: I would argue that women (and other minorities) often make more progress when they stopped behaving all polite and quiet, and start getting angry. This doesn’t have to mean “frothing-at-the-mouth” levels of anger, but I think that a degree of “we are not going to be bought off with polite words any more” is often essential for change.


  2. Nice post. This whole thing seems like a giant uncertainty/threat manipulation. Outgroup denigration and woldview defense being replicated in real time lol. Regardless of how bad/good things are, it seems to me that everyone can agree on more vigilant research methods moving forward (pre-registration + combatting publication bias). 🙂


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s