SORTEE two-year anniversary: our members’ voices

Between June and September 2021, we started the #SORTEEvoices blog series by asking inaugural members to choose a few questions to answer from a list of 30 options (15 questions about open science, reproducibility, and transparency; 15 miscellaneous questions)¹. Responses from sixty-four inaugural members were posted on our blog every week until October 2022. To celebrate SORTEE’s 2-year anniversary in December 2022, we’d like to look back and summarise our members' voices.

Who answered?

Most members who answered our questions were academic researchers: 38% PIs (professors, researchers, lecturers), 33% postdocs, and 20% graduate students. PIs were overrepresented in this blog series compared to our full list of members which skews more towards early-career researchers (in 2021 and 2022). We also had a couple of members of NGOs and of research support staff.

Which open research questions were the most answered?

The answers to the four most answered questions provided inspiring stories of personal paths toward the adoption of open research practices, as well as some interesting perspectives on areas for improvement in members' research fields.

How did you become interested in open research?

Many members felt lucky to have been exposed to the subject early on (team, mentor, teacher), or to have met like-minded people. At SORTEE, we work to create a welcoming environment for exchanges and to give a sense of community to our members, including through our conference and our Slack workspace. We hope to continue to foster these connections for many more of our members going forward.

Many members mentioned particular events that made them start caring about open research. For example, several shared their experience of not being able to access research outputs (e.g. articles, data, or code) that they needed for their work. Some lost access after moving out of academia (e.g. to work at NGOs), while others had limited access to articles if they worked at a non-wealthy institution. Several members also reported struggling to replicate others' findings, or to publish null results, and a couple of members even witnessed cases of scientific misconduct. SORTEE’s main goal is to promote higher standards of good research practices in the fields of ecology and evolutionary biology, by providing training and building a community of practice, notably through our webinar and workshops series and our participatory events during our conference.

A few other members mentioned they read about the replicability crisis in other fields (e.g. psychology or medicine), came into the open research movement from a philosophy of science background, or realized more generally the current academic research culture and its incentive structure is broken. Finally, one person mentioned the philosophical implication of meta-analyses. With SORTEE, we plan on engaging in advocacy work with stakeholders across the sector to reform the design of incentives and policies; we have the ambition to similarly inspire other fields to join the reform movement; and overall, we strive to increase the impact of research by making outputs open and reusable for all.

What open research practice did you adopt that you found most useful?

The winning practice in terms of usefulness for the adoptees was version control (for code and sometimes data), for instance through Git and GitHub. Our members also praised the usefulness of the Open Science Framework (OSF) to share their research outputs. Finally, some members highlighted some specific programming skills (e.g. RMarkdown) to increase the comprehensibility of their codes for others, the use of preprints as a way to share their articles in an open access format, and the adoption of preregistration to carefully plan their studies prior to conducting them.

With SORTEE’s webinar and workshop series, we will strive to raise awareness and teach these skills that were found most useful!

If you had the power to change one thing in your field, what would it be?

One member wished for more and better statisticians to be involved through the lifetime of projects, and another wished that journals across the board would implement reporting guidelines for authors to share statistical information so as to be reusable in secondary analyses such as evidence synthesis.

All the other members who chose to answer this question deplored the current academic research culture and particularly the current ‘publish or perish’ reward system. Members expressed their desire to perform careful, nuanced research as opposed to racing to publish first. A shift towards publishing more Registered Reports could help researchers be rewarded for the robustness of the experimental design and quality of data. Several wished to further reform the publishing system, for instance using a GitHub-like system with community discussions, to continuously track and log updates of research outputs and document how results change as more data or better methods become available.

Which open research practice do you think deserves more attention?

There were mainly two categories of open research practices that were thought to deserve more attention.

1. Code and data sharing such that they are discoverable, checkable, and reusable. One member also acknowledged that this requires rewarding the production of these additional research outputs.

2. Pre-registration, particularly in the form of Registered Reports, to eliminate (conscious or unconscious) p-hacking, confirmation bias, HARKing, selective attention bias, publication bias, and incentivise addressing interesting questions with rigorous methodology as opposed to incentivising finding sexy-results.

Again, with SORTEE’s webinar and workshop series, we will strive to raise awareness of these practices deserving of attention!

The most answered personal question was: Where were you born and raised?

This showcases some of the diversity of background within our community, an aspect we take pride in!

Given the interest in these blog posts and personal stories, we are launching a second wave of ‘interviews’ with our newest members! Choose from a pool of new and old questions to respond to and contribute your thoughts on open science/reproducible research and/or tell us more about yourself! Take a few minutes to fill out the #SORTEEvoices form.

You can become a member here: https://www.sortee.org/join/. Find all our interviews at https://www.sortee.org/tags/member-voices/.

Acknowledgements

Summary blog writer: Malika Ihle

Q&A survey distributor: Kaija Gahm

Blog post editor: Colin Strine and Rose O’Dea

Blog series instigator, maintainer, and advertiser: Rose O’Dea