Welcome to the IDEA Lab's Reviewer Hall of Shame! Peer-review can be enlightening, critical, brutal. But sometimes, it can also be completely terrible, failing to meet the standards of peer review (well at least based on the norms and guidelines we ascribe to in the IDEA lab). Below are some examples for your entertainment...
Embedded Files
Paper Title: Concern for Future Generations Predicts Present-Day Extraordinary Altruism: A Case Study of Organ Donorship
Journal: European Journal of Social Psychology (May 2025)
Reason: Well, technically this is more of an editorial decision, but this was the 3rd time in my submission history to EJSP, where a paper received only one peer review, which requested major revisions, that the editor ultimately rejected...
Paper Title: Presidential Candidate Endorsements by Scientific Journal Decrease Trust in Science Especially for Moderate and Conservative Americans
Journal: Personality and Social Psychology Bulletin (May 2025)
Reason: Commenting about p values that are below .05 and noting that they are not low enough as evidence of the significance of the work, when all three studies were pre-registered with a-priori power analyses
Actual Comment: "I appreciate that the authors noted which DVs in study 3 were exploratory. But overall on pre-registered tests, effects are not as strong (p values often hovering around .05; with conspiracy/covid-19 beliefs not being significant), as they are sometimes described throughout the manuscript (e.g., abstract)"
Paper Title: The Motivation to Protect Future Generations as a Source of Meaning and Mental Well-Being
Journal: Personality and Individual Differences (May 2025)
Reason: This reviewer made a record-breaking 114 comments! (a crime of its own). A good amount of them contained false information, failed to acknowledge information that was in the paper, assumed our intentions, commented on work unrelated to this paper, and in several cases made ad-hominem attacks to the research team.
Actual Comments: "1. I seem unable to link virtually anything contained in the submitted paper with any form of recognizable reality. Everything discussed seems so abstract, vague, generic, hypothetical, and even metaphysical. In many ways, the paper seems an exercise in speculation given all the uses of such words as "may," "can," "might," and "could". After all, ANYTHING "could" happen.
5. There is an implicit null hypothesis in every submitted paper that takes the form of "the authors do not know of what they speak and are thus not fully qualified to advance knowledge".
6. Give your readers enough evidence so that they can REJECT the null hypothesis. Enlighten your readers. Tell them something they may not have known. Link the writing with recognizable reality. Interpret findings appropriately. Demonstrate enough knowledge of the literature, with enough specificity, so that readers can be totally confident that the authors know their stuff.
7. Believe it or not, reviewers generally WANT to recommend that a paper be accepted for publication, but they really DO need a reason to reject the implicit null hypothesis.
8. When authors seem unable to write clearly and link their writing with the real world, and when they seem unable or unwilling to interpret results correctly, readers will be unable to reject this hypothesis.
"Concern about Future People" Is Not Real
9. If someone in the real world WANTED "to protect future generations," what would they actually be DOING? Provide a specific and vivid example. Then provide three more.
10. If someone wanted to HARM "future generations," what would THEY need to do? Fire nuclear warheads? Poison the water supply? Burn down forests? Wouldn't any of this harm people RIGHT NOW, with the direct and immediate harms outweighing any possible lingering harms centuries from now?
11. Consider concrete examples of "prosocial behavior" (more examples than appeared in the paper).
--No one volunteers at a soup kitchen to help people who will not be born for a millennium.
--No one makes every effort to reduce garbage and to recycle whenever possible to help unborn people.
--No physician associated with "Doctors without Borders" thinks even in passing about what the world will be like 100,000 years in the future.
--No one that helps build homes for Habitat for Humanity cares about what might happen 10,000 years from now.
12. This is reality, and reality is not optional. If the authors disagree about this version of reality, then interview such people in depth and find out directly.
28. Imagine two people. One has a "deep concern". The other has a "shallow concern". Without guessing, identify the one with "deep concern".
29. Imagine two other people. One has a concern for ALL future generations. The other has a concern only for SOME future generations. Observe them. Don't guess. Which is which?
30. We may as well discuss how many angels can dance on the head of a pin, or how elves, leprechauns, and pixies should be treated if we assume that they exist. If there are ANY real world implications of ANY of this, the authors are obligated to provide these. It is not the job of readers to make connections FOR the authors.
31. Although the authors seemed reluctant to provide their readers with relevant information, it may be worth noting some crucial alterations that transpired between the recent Syropoulos et al. (2025) paper and the submitted paper.
35. Consider what the literature review reveals about "Legacy Motivation" (the first two key words). On the middle of page 4, readers learned that the paper "draws upon" research into this (as well other unexplained phenomena). Please get to the point.
36. On the bottom of page 7, readers learned that "emerging research" into "Legacy Motivation" exists. Readers do not need to know this, as there has ALWAYS "been research". Five full pages of text, with nothing specific said about "Legacy Motivation".
41. In the second paragraph on page 8, the authors reveal that not much is known about "Legacy Motivation". Apparently, we know more about the impact of unexplained "generativity" than about "Legacy Motivation". Readers do not need to learn THIS.
45. This sentence that conveys no meaningful information reads as follows: "Even outside the context of stressors linked directly to future-oriented challenges [perhaps use the words "climate change" to be specific rather than generic], legacy motivations [one sentence earlier, there were "legacy CONCERNS"] show potential benefits for well-being".
50. If someone WANTED to help others 1000 years in the future, please specify what they would need to do SPECIFICALLY that wouldn't also help people RIGHT NOW.
55. A paper determined to get to the point might not begin this sentence with SEVENTEEN unnecessary words (and an unnecessary cited work). The last word in the eighth line is "may". As before, ANYTHING "may" happen.
65. Three additional items require guesswork on the part of readers. Please never compel readers to have to guess.
71. This seems to mean whatever the authors want it to mean. Specific measures are used with ZERO discussion about nomological nets surrounding these underlying constructs.
80. One noteworthy descriptive statistic common to TOO MANY measures involves unusually high coefficient alpha estimates.
84. What makes "life satisfaction" (α = .92) a measure of "mental wellness"? Occam's Razor suggests that we call things what they ARE rather than what they are NOT.
89. If someone agreed with "I am a good person and live a good life" (a "flourishing" item), they could not POSSIBLY disagree with "I want my life to impact others in a positive way".
99. The measure of "impartial intergenerational beneficence" (α = .98; POINT NINE-EIGHT) is simply not interpretable. High scores in no way represent "concern" despite author claims on pages 4, 18, and 26. These also do not represent a prosocial worldview despite what it says on page 11. Material changes in procedures between studies were noted earlier.
101. Each one of these items was followed by "1000 years in the future" as well as three other even more absurd and metaphysical time frames (10,000 years in the future, 100,000 years in the future, and 1,000,000 years in the future). See Study 2a, Syropoulos et al., 2025.
109. Peoples' responses to metaphysical phenomena reveal nothing meaningful. Just like asking them whether it is IMPORTANT to consider long-term consequences that will not be realized for ONE MILLION YEARS. Not everything can be important.
110. Perhaps the reviewers of the paper appearing in Personality & Social Psych Bulletin (and the editor that accepted it for publication) were not at all curious about construct validity. Those reviewers, however, are elsewhere. What construct do YOU believe is assessed here? Then, explain this using the actual items.
111. It would be possible to continue in this vein. For example, kindly explain "future self-continuity" even a little bit. What does it even mean to be "connected to" the self a quarter century from now? Kindly explain precisely how this isn't metaphysical.
112. Please consider eliminating all examples of "word salad". Consider the top of page 5:"A leading explanation for this phenomenon [donating an internal organ was mentioned in the previous sentence--is THAT it?] is that helping others contributes to a sense of meaning in life [or maybe it simply helps others] by connecting people to something larger than themselves [such as helping others (without violating Occam's Razor)?], activating a [unobservable and metaphysical] self-transcendent process that in turn inspires [unobservable and metaphysical] self-flourishment--a form of eudemonic well-being [these words explain nothing even though they "sound good"] characterized by fulfillment and actualizing one's potential". A cited work, by itself, is inadequate. The job of AUTHORS is to explain to THEIR readers.
114. Please direct your considerable creativity in a direction likely to advance meaningful knowledge."
Page updated
Google Sites
Report abuse