Open Science

The Replication Crisis is not a crisis for researchers - it is a crisis for society

Perverse incentives in research careers lead to poor research practices prevailing. This problem may not necessarily be a problem for researchers as their careers can benefit from questionable research practices. The end users of science (such as government agencies, policy makers and wider society) are the ones who are negatively impacted by poor science and we argue here that the systemic change must come from them and not from within the research community alone.

Continue reading

Open Science – what’s the way (for Australia)? – Notes from a panel discussion

[This post has been originally posted on ecoevotransparency.org]

We all heard about Open Science, and particularly Plan S, which has been announced in Europe last year (read more here). On 14th February 2019, I had an opportunity to be a panelist during discussion on what it all could mean for Australia. The panel discussion was organised by Springer Nature as a part of the ALIA conference, which is the main meeting for the librarians and information specialists in Australia and New Zealand (I realised these are mostly lovely middle-aged ladies, although they said more men are starting to join this profession with the new technologies, closing the “gender gap”).

Continue reading

EcoEvoRxiv launched!

[This post has been originally posted on ecoevotransparency.org]

I am very excited to announce the launch of EcoEvoRxiv – a preprint server where ecologists and evolutionary biologists can upload their forthcoming papers. I am aware that many ecologists and evolutionary biologists already use the preprint service, bioRxiv and that’s great! I have used bioRxiv several times myself. EcoEvoRxiv is a more targeted server, and it is convenient because a preprint at EcoEvoRxiv can seamlessly integrate a project that makes use of the services at the Open Science Framework (OSF). My group (i-deel), like others, uses OSF for project management so this is a great feature of EcoEvoRxiv.

Continue reading

Join the Credibility Revolution!

[This post has been originally posted on ecoevotransparency.org]

Last week (14-15 Nov, 2018), I went to Melbourne for a workshop (“From Replication Crisis to Credibility Revolution”). The workshop was hosted by my collaborator and “credibility revolutionary” Fiona Fidler.

I suspect many workshops and mini-conferences of this nature are popping out all over the world as many researchers are very much aware of “reproducibility crisis”. But what was unique about this one is its interdisciplinary nature; we had philosophers, psychologists, computer scientists, lawyers, pharmacologists, oncologists, statisticians, ecologists and evolutionary biologists (like myself).

Continue reading

Prior probability and reproducibility

[This post has been originally posted on ecoevotransparency.org]

Physicist Carl Sagan famously said “Extraordinary claims require extraordinary evidence.” I think its useful to extend this to the distinctly less elegant “surprising findings are less likely to be true, and thus require a higher standard of evidence.”

I started thinking more about what influences the reliability of a scientific result when analyses of my post-doc data weren’t lining up with published findings from other studies of the same species. When I encountered this problem with reproducibility, the causes I first focused on were old standbys like multiple tests of the same hypothesis driving up type I error and the flexibility to interpret an array of different results as support for a hypothesis. What I wasn’t thinking about was low prior probability – if we test an unlikely hypothesis, support for that hypothesis (e.g., a statistically significant result) is more likely to be a false positive than if we’re testing a likely hypothesis. Put another way, a hypothesis that would be surprising if true is, in fact, less likely to be true if it contradicts well-supported prior empirical understanding or if it is just one of many plausible but previously unsupported alternate hypotheses. Arguments that I’ve heard against taking prior probability into account are that it isn’t ‘fair’ to impose different standards of evidence on different hypotheses, and that it introduces bias. I think the risk of bias is real (we probably overestimate the probability of our own hypotheses being true), but I think the argument about fairness is misleading. Let’s consider an example where we have a pretty good idea of prior probability.

Continue reading

An iconic finding in behavioral ecology fails to reproduce

[This post has been originally posted on ecoevotransparency.org]

Just how reproducible are studies in ecology and evolutionary biology? We don’t know precisely, but a new case study in the journal Evolution shows that even textbook knowledge can be unreliable. Daiping Wang, Wolfgang Forstmeier, and co-authors have convinced me of the unreliability of an iconic finding in behavioral ecology, and I hope their results brings our field one step closer to a systematic assessment of reproducibility.

Continue reading