I have to admit: I am more than a bit nervous to write this post. Many “big” events have transpired within the last week of the ongoing discourse regarding replicability in psychological science, and the resulting exchanges on Twitter, Facebook, and the blogosphere have seemed incredibly heated and personal. Most centrally, the widely-discussed (and embargo-leaked) commentary critiquing the Reproducibility Project was released in Science:
Only to be rapidly responded to with–err, preceded by–thoughtful critiques of the critique (and even critiques of the critiques of the critique) by the Reproducibility Project, and many others:
In the midst of all of the online activity, I have seen some pretty ugly behaviour, from one-off snide comments, to elaborate flame wars, including (but not limited to): name-calling, mean-spirited poetry, attempts to shame individuals from participating in the discourse, and appeals to authority intending to silence critical discussion. This is not the level of “scientific”communication that our discipline deserves–least of all now. And I felt like I needed to blog about this recent trend in online communication about replicability, because I actually feel hurt for many of the individuals involved, many of whom I hardly know at all.
For me, Simine Vazire comes close to hitting the nail on the head with this tweet on how psychologists should be conducting themselves in the public replicability discourse:
However, I would go one step further, and remove many of the qualifications; these rules shouldn’t just apply to those who are conducting replications, failed, successful or anything in between. Instead, these rules could and should also apply to anyone wanting to partake in the replicability discourse, no matter the side. Indeed, there has been a lot of bad communication behaviour to go around on both sides of the replicability aisle.* Dan Gilbert, for example, previously occupied the position of offering mean-spirited commentary when referring to those interested in replicability as “second-stringers”, and now finds himself the target of a considerable amount of online vitriol. I lament both sides of his exemplar.
So why, exactly, is civility so important to public scientific discourses, like issue of replicability ?
- Civility is inviting; mean-spiritedness is repelling. My interpretation of the purpose for the #OpenScience movement was to get more/all scientists onboard with becoming more transparent about their data, methods, materials, and analyses. Call me a closeted self-determination theorist, but I believe that the surest way of accomplishing this goal is for the #OpenScience community to be warm, inviting, and encouraging of people making changes to the way they conduct their science–not to bludgeon them into it with name-calling or professional threats/guilt-trips.Perhaps some feel as though offering mean-spirited comments targeting prominent people in the field is the only way to get these prominent people to change their scientific practices to become more Open Science friendly .** But even if that was for sure the case–and I am skeptical of the efficacy of this strategy–such a strategy can have chilling effects on third-parties who are impacted by the replicability discourse, namely, junior colleagues. An unnamed junior colleague of mine, for example, recently disclosed to me:
…right now I have a study I didn’t cook at all that has N = 225 and p = .048, and I don’t want to publish it cause if someone took their [replicability] ray gun at me, I would look suspicious.
This kind of fear hurts both the field–which misses out on this research–and my junior colleague–who can scarcely afford to avoid publication attempts, irrespective of how “borderline” their effects are. That my junior colleague insisted on remaining anonymous for their protection (because they are that scared of being called a “hack”) only serves to further highlight the repelling fear of mean-spiritedness.
- Civility leads to focus on the actual issues; mean-spiritedness distracts from the actual issues. Sanjay Srivastava aptly characterizes the situation:
In other words, when we are not civil, our audiences will focus their reactions on our mean-spiritedness–not the substance of our arguments. Sometimes these reactions may happen honestly/automatically, whereas other times they may be a more deliberate attempt to create a straw-man counter-argument. Either way, the end-result is the same (so let’s not waste time trying to distinguish cases on this basis): less civility = less engaged discussion on the arguments that matter. For example, implying that a critic–be them “pro-replicability” or “replicability-skeptic”–of our position is a hack (on whatever subjective/quantitative basis), and therefore their critique shouldn’t be attended to, doesn’t actually address the substance of their critique. And I’ll espouse the bold position that I think few people in our discipline are that self-serving/stupid/mindlessly guild-promoting/evil, and therefore don’t deserve to have their voices silenced in this dismissive way.
- Civility is a positive experience for both parties; mean-spiritedness is a negative experience for at least one party. I have had some riveting debates regarding replicability-related issues with many of my colleagues, and some of the most memorable happened with instructors and colleagues with whom I disagreed the most about replicability-matters. But the funny thing about these disagreements is that they were also incredibly enjoyable–arguably more enjoyable than discussions with colleagues who shared my beliefs– in large part because my counterparts and I argued passionately, but respectfully. I’ve also noticed that neutral or undecided third-parties enjoy such debates more when they are conducted with civility. Mean-spiritedness, alternatively, guarantees to leave at least one party (if not all) will leave feeling crappy. And there are already more than enough negative-affect-inducing experiences in our profession: our papers and grants are frequently rejected; we are always under the gun of some professional hurdle, be it passing defenses/getting jobs/getting tenure/etc.,; and we receive constant critical commentary about our research with massive delays until our personal rewards are realized. In other words, academic life is enough of a struggle already, without having to deal with colleagues being mean-spirited to one another in public domains.
Despite my handwringing about civility, I actually do think there is a time and/or place for mean-spiritedness. Believe it or not, I get the impulse to run someone down, or make a snappy and disparaging remark about a paper/blog-post/tweet, etc., especially when someone runs amok of my scientific/political values.*** Such responses help us to blow off much-needed steam, generate clicks for our blogs, and help us to feel closer to like-minded individuals. But at the end of the day, civility is a surer path than mean-spiritedness, for our goal of bringing others into the family of open and replicable science. So if you come across an article/story/blog-post/tweet that gets your goat, save the funny, but mean-spirited commentary for your private exchanges with like-minded friends. A civil approach to the public discourse about replicability, instead, will facilitate invitation and inclusion, a focus on the actual issues, and a more positive experience for all–exactly the type of discourse our science and its members deserve.
Is It True? Is It Necessary? Is It Kind?
Oh! Stay, dear child, one moment stay,
Before a word you speak,
That can do harm in any way
To the poor, or to the weak;
And never say of any one
What you’d not have said of you,
Ere you ask yourself the question,
“Is the accusation true?”
And if ’tis true, for I suppose
You would not tell a lie;
Before the failings you expose
Of friend or enemy:
Yet even then be careful, very;
Pause and your words well weigh,
And ask it it be necessary,
What you’re about to say.
And should it necessary be,
At least you deem it so,
Yet speak not unadvisedly
Of friend or even foe,
Till in your secret soul you seek
For some excuse to find;
And ere the thoughtless word you speak,
Ask yourself, “Is it kind?”
When you have ask’d these questions three—
Ask’d them in all sincerity,
I think that you will find,
It is not hardship to obey
The command of our Blessed Lord,—
No ill of any man to say;
No, not a single word.
*Full disclosure: I identify as a “concerned citizen-scientist”, who believes the replication “crisis” in psychology is real, and that there is much we can do to begin improving the quality of the science we produce. I have even written a couple of articles about possible solutions for psychology, am attempting to bring the “good-word” of replicability concerns to other science communities that I am a part of, and am trying to make Open Science practices an increasingly large part of my own research. And though I feel like I shouldn’t have to disclose all of this, in order to be taken seriously, I anticipate that I might need to…
**I have also seem some commentary to the effect that mean-spiritedness is justified because of power disparities between those who are espousing and using Open Science practices, and those who are not. Without getting into the matter in more detail, I think in this particular case, this argument can be a specious barrier offered by some to avoid challenging themselves to a higher standard of civil discourse. Many among the vocal “pro-replicability”crowd, for example, are men**** occupying tenure-track positions at R1 or R1-ish schools. And many of the vocal “replicability skeptics” crowd are also men occupying tenure-track positions at R1 or R1-ish schools. Might there be a relative prestige/power differential here? Perhaps. But by any absolute standard, members of both groups occupy highly privileged positions.
***Trust me, I have a pretty strong pun/haiku/limerick game when it comes to the stuff I disagree with…
****This observation is not meant to overlook the vocal female contributors to the discourse, many of whom (e.g., Samine Vazire, Liz Page-Gould, Erika Salomon, etc.,) who, in my opinion, are some of the most consistent sources of wisdom within the replicability discourse. Rather, much of the mean-spiritedness I have observed has been carried out by men. So when I see appeals to power differentials to justify this conduct, well… (see **)
[I am cyynically awaiting my r-index/incredibility index/p-curve]