#15: 🚜 Overhauling the Peer-Review
#15: 🚜 Overhauling the Peer-Review
Publishers require independent researchers to assess a manuscript’s originality, validity and significance for free
Welcome! If you haven’t subscribed, join this curious group by subscribing here:
💬 In this note:
🚜 Overhauling The Academic Peer-Review Process
💪🏼 Strength Training for Longevity
🧱 Achieving Lasting Change: The Importance of Small, Consistent Efforts
📚 Atomic Habits
🚜 Overhauling the Academic Peer-Review Process
→ This is part 2 of a 3 part rant about the scholarly publishing industry. Want to read Part 1? Check it out here.
What is Peer Review?
Scientists share their research by submitting their work to journals for publication and dissemination. After submission, the next step of the publication process is the “peer-review,” which is getting the research reviewed by experts in the field with similar competencies as the producer of the work.
Why?
Publishers rely on peer-review to assess a manuscript’s originality, validity and significance, and if the manuscript is worthy of being published. The Publisher, Elsevier, says that,
Reviewers play a pivotal role in scholarly publishing. The peer review system exists to validate academic work, helps to improve the quality of published research, and increases networking possibilities within research communities. Despite criticisms, peer review is still the only widely accepted method for research validation and has continued successfully with relatively minor changes for some 350 years.
350. years. old.
That means the Peer Review system is older than the USA. I can just imagine a bunch of scientific-revolution bros gathering in a basement lab in the 1670s and saying “Hey, want to read my paper, and I’ll read yours? And then we print them?”, and the Peer Review was born, along with its many issues.
Reviewers Are Not Paid
Publishers charge researchers thousands of dollars to publish their research (Re: rant Part 1), and then they require reviewers to do many hours of work for free to review the science in the manuscript. All the while, the publishers make enormous profits. Which has led to a debate around whether or not peer reviewers should get paid.
The pro-payment movement believes that incentivizing researchers would result in more reviewer availability and argues that a $450 fee would be reasonable for for-profit publishers to pay per peer review.
The anti-payment team believes that paying reviewers would lead to biased reviews and unsustainable costs on the publishers.
While this debate continues, scientists continue to review for free because they feel a sense of duty to help advance their disciples, as well as the need for reciprocity, knowing other researchers volunteer to peer review their manuscript submissions.
Peer Review Is Biased
Scientists have some control over who reviews their papers. If there is a conflict of interest, or two researchers dislike each other, the author of the paper can ask the editor to exclude those people from the review.
Essentially the publisher makes it possible for the author to exclude the people who will say “No” to their submission. Additionally, researchers often recommend others in their field to review their own work, and can create an environment where the manuscript gets published.
Not too much different from the visual I painted above - it is possible to get your friends to review your paper in order to get it published.
More Publications = More Prestige
The researchers who are publishing the most are the ones who are benefiting most from the system. They gain prestige by having many publications, and this prestige makes them attractive to reviewers when applying for large government grants. With large grants and more resources, the researcher can produce more work and publish in prestigious journals again.
Researchers just starting their careers, ones with limited resources, or ones working at less reputable universities have a difficult time making a name for themselves. In order to push papers through some of the big journals, these scientists need big resources behind them - time, money and people, which they do not get until they have the prestige, which ends up being a chicken and egg problem.
Many Revisions
After a manuscript is reviewed, the reviewers can come back with feedback and ask for revisions. Oftentimes these revisions are not editorial, but scientific. The reviewer will ask for more experiments to prove the findings in the paper, leading to many months, or even years of work to generate the data for re-submission. There are cases where reviewers can bully scientists to make they do more work for nothing wrong with their submission.
Closed-Door Reviews
Most of the time reviews are not published, and they are kept confidential.
In an effort to eliminate bias, reviews are typically a ‘single anonymized review’ meaning the reviewer’s name is hidden. Sometimes the author’s name and the reviewers name are hidden, a ‘double anonymized review.’
However, hiding the reviewer’s names and also ultimately keeping the review confidential does not always eliminate bias, this can lead to harsh reviews under the anonymity.
The alternative to this is the ‘Open Review’, where both the reviewer and author are known to each other, which has been met with mixed reviews.
Many believe this is the best way to prevent malicious comments, stop plagiarism, prevent reviewers from following their own agenda, and encourage open, honest reviewing. Others see open review as a less honest process, in which politeness or fear of retribution may cause a reviewer to withhold or tone down criticism.
In an effort to make peer review more transparent and accountable, between 2014-2017, 5 journals owned by publisher Elsevier experimented with publication of peer review reports (signed or anonymous) as articles alongside the accepted paper. Interestingly, only 8% of reviewers agreed to reveal their names.
Who’s disrupting the 350 year old peer-review process?
1️⃣ The Longevity Decentralized Review (TLDR) founded by Tim Peterson and backed financially by VitaDAO is an on-demand peer review service where they pay reviewers cash for their reviews.
2️⃣ ResearchHub, founded by Coinbase Founder Brian Armstrong, as a sort-of-GitHub for science with utility behind the paper. ResearchHub has hired 60 editors who are qualified to review scientific papers. The editors are paid in ResearchCoin, which allows governance rights on the platform. Currently they are seeking preprints or draft research manuscripts to receive peer reviews.
3️⃣ DeSci Labs believes that a decentralized peer review system can be achieved by “autonomous research communities of the best researchers in every field, who are incentivized and rewarded for providing open, timely, and high-quality peer review and who select the most important contributions to be highlighted in a transparent way.”
💪🏼 Strength Training for Longevity
Muscle mass and strength can be a key indicator to how long you will live and how healthy your life will be.
If you’re over 65, there is a one in four chance that you will fall. Strength training can prevent falls by helping to restore balance and improve mobility. Increased muscle mass can also help to speed up recovery after illness or injury.
This article published in the British Journal of Sports Medicine earlier this year states that muscle-strengthening activities can reduce one’s mortality risk by 10% to 17%.
The researchers set out to find out how much strength training we really need to support longevity. They analyzed 16 existing studies related to exercise, some of which had over 480,000 participants. They found that strength training was associated with greater cardiovascular health outcomes, and in turn, greater longevity.
Additionally, they realized that it didn’t require too much exercise to see those benefits. Just 30 to 60 minutes of muscle-strengthening exercise per week can have a positive effect on heart health, and support a long and healthy life. More than one hour of strength training does not seem to increase your health benefits.
Coupling strength training with aerobic exercise resulted in even greater positive effects for heart health and longevity.
🧱 Achieving Lasting Change: The Importance of Small, Consistent Efforts
The new year is just around the corner and that’s got me thinking about New Year’s resolutions.
Most people fail to keep their new year’s resolution because they are treating a marathon like a sprint. Many people have this idea that “I want it all, and I want it NOW!” Which dooms them for failure. They start to exercise, and if they don’t lose weight in the first week, they give up.
Over 80% of people fail to keep their new year’s resolutions and I want to let you, loyal Note reader, in on a little secret to keeping your New Year’s resolution, and ultimately, changing your life in just one year…and that secret is,
The Power of Tiny Gains.
Slow and steady is the key to making changes, and those changes operate by Power Law. If you are just 1% better every day, you will improve by 38x by the end of the year! However, compounding works both ways. If you’re getting 1% worse everyday, you’ll be in a lot of trouble at the end of the year.
Doing something and sticking to it, and measuring that progress is a great way to keep you motivated. For example, I started writing this newsletter 15 weeks ago! Can you believe it!? My writing has improved, I’ve gotten into a good habit, and I now can consume articles and media in a much more productive and meaningful way. I also do not relate at all to the person who wrote Nina’s Notes #1. But I do thank her for getting me started!
Sahil Bloom explains on his blog the steps to get started to make those daily incremental gains and truly change your life. He says the 4 steps to continuous improvement are:
Establishing the desire
Create a plan
Execute the plan
Track and adjust
Establishing the desire
When you have the desire to make progress at something, you’ll find the energy and motivation to make it happen. Write down exactly what you want to do, and put it somewhere you can see it.
Create Your Plan
This means doing the thing you want to improve every…damn…day. For example, if you want to improve your writing, write everyday. If you want to improve your relationships, practice being present and spending time with those people, every single day.
Execute your Plan
Do your daily action for 30 minutes per day for 30 days straight. Or, what works for me particularly well is setting 1-2 hour blocks of time per day for “deep work” where the work is completed without distraction. That means, no taking your phone with you to the gym class. It means, turning off all apps and notifications while writing for 1-2 hours.
Track and Adjust
Use a calendar to mark a big X on the days where you did your exercise, or your writing practice, or whatever practice it is you are doing. Make it visual. Set goals and share them with your friends and ask them to check up on how you are doing. Create a google sheet to track your progress and share it. Allow yourself the grace to adjust along the way as needed. One skipped day is not a problem, it’s when that skipped day turns into a skipped week…skipped month. Be kind to yourself and allow yourself the day off, but get back to it the following day.
It’s hard to see your progress in the moment, and sometimes it takes 3 months to be able to reflect and say “wow! Look at how far I’ve come.” This is where tracking can help - it can help you zoom out and see where you started. It’s also good to reward yourself when you hit a small milestone.
It’s truly amazing how much change can occur within one year. The possibilities are endless.
What are you planning for 2023? What change do you want to create in your life? Comment or reply to this message - I’d love to hear it and cheer you on!
📚 Book of the Week
Atomic Habits by James Clear.
If I haven’t convinced you now that making small, daily improvements, rather than large step-function leaps can transform your life. Then I’ll let author James Clear spell it out in his book, Atomic Habits. Clear perfectly illustrates the power of making super small, atomically small, improvements each day in order to progress and grow.
⚡️ Check this Out
Pantone’s 2023 color of the year is “Viva Magenta.” It’s the pink we didn’t know we needed, and it is “vibrating with vim and vigor.”
Two articles on the failure of peer review in (almost) one week: https://open.substack.com/pub/experimentalhistory/p/the-rise-and-fall-of-peer-review?r=1fmmk&utm_medium=ios&utm_campaign=post it must be Christmas!