TL;DR

• Protocols.io – a collaborative, open access platform to share scientific recipes.
• Scientific reproducibility is likely no worse than 30 years ago, but because of increased scrutiny and better technology now available, we are getting better at addressing it.
• In traditional peer review, reviewers don’t verify the working of protocols. But on protocols.io, the community has a chance to directly run the protocol and confirm its validity or raise questions and concerns.
• Having access to tons of great protocols saves scientists tons of time and this ever-growing list of protocols can dramatically help speed up biomedical research
• Over the long run, the platform may become a more meritocratic way to showcase reputation amongst scientists.

Lenny Teytelman is the cofounder and CEO of protocols.io – a platform which allows to share research protocols among scientists. He is a geneticist as well as computational and experimental biologist who did his PhD at Berkeley and postdoc at MIT. He is very curious about many questions in basic science and passionate about open science.

How did Lenny come up with protocols.io?

When Lenny started his postdoc at MIT, he thought he would spend 6 months and have amazing new data. He went into the literature and found a method that he thought he could use out of the box to conduct his experiments. However, something in that method was broken, and he needed 1.5 years to find out which step in the protocol it was and how to fix it. He knew other researchers were facing the same problems frequently as well. So when his friend and now cofounder & CTO Alexei Stoliartchouk approached him about building an app for biologists, they decided to start a company and became obsessed with improving reproducibility by creating a place where scientists can publish, discuss and update protocols.

How does protocols.io work?

Immediate sharing

Protocols.io allows researchers to access and publish scientific recipes or protocols. When you publish a protocol, you get a DOI (a digital object identifier, a persistent identifier used to uniquely identify objects, used heavily in scientific publishing) and you don’t have to wait for publishers and editors to approve your work before you can share it with the wider community.

Protocols.io itself doesn’t try to reproduce the protocols. As in peer reviews in the traditional publishing process, where the reviewers also don’t reproduce the protocols in their labs. They assume it works, so there is no guarantee that they work.

Decentralized community collaborates to validate & improve protocols

In protocols.io, the community of scientists tries the protocols. They can share success of reproduction, thus further adding credibility to the protocol as well as the researcher. Or they can ask questions about the protocols and share difficulties or improvements with the authors and other readers. They can also ‘fork’ a protocol working on say, one bacteria strain, and try it on a different strain. Thus, protocols can quickly be validated, improved and tried out in new environments.

Gaining Adoption

11,000 scientists are now registered to protocols.io and are sharing protocols regularly, to the tune of more than hundred new public protocols a month (among with several few hundred private ones). Many more are visiting the site regularly to get inspired. The initiative is also growing thanks to 20 ambassadors who are volunteering for outreach and for input to improve the platform and community management – these are students and postdocs across 5 continents.

Does protocols.io help with the reproducibility crisis?

Lenny wonders if we have any evidence that our reproducibility is worse than 30 years ago. He talked about this issue with Richard Harris – (an NPR science journalist, who published a book Rigor Mortis on how biomedical research is difficult to replicate because of bad experimental design) and concludes that now we simply observe enhanced visibility to the problem of reproducibility, a problem we’ve always had. However, now in the age of Internet and technological advancement, definitely we as research community should do much better.

Publishers do really care

The first publisher who decided to collaborate with Procolos.io was the Genetic Society of America and currently there are many other publishers – over 200 journals, including PLOS One. Lenny’s message is that many publishers do really care about improving reproducibility of results. Both open access and subscription publishers collaborate with Protocols.io to encourage scientists to add to their papers comment that for details you can see Protocols.io. This is able to make research papers more reproducible.

How does protocols.io impact researcher’s lives?

Public Protocols can become an additional source of reputation amongst papers

When someone forks a protocol and creates a new version for their setup and lab, the original person’s credit is preserved along with the new author.

There are already incentives in academia to share detailed methods. For example, researchers are starting to include sharing on protocols.io as part of their NIH and NSF grant applications.

Saving time

There are a lot of incentives to make research more reproducible, but scientists are busy. Being reproducible takes time. In the beginning, uploading your protocols to protocols.io can be time consuming. Over time, as more and more protocols get uploaded, we might reach a point where most of the protocols you need in your daily life are online and it will start saving you tons of time.

What is the business model behind protocols.io?

If you are a group and want to keep your protocols organized in a private mode, protocols.io charges a per user subscription fee. Protocols.io also makes money from certain anonymized analytics around research recipes and this is a bigger revenue source right now.

Generous grants from funders like the Gordon & Betty Moore Foundation, the Open Philanthropy Project and the Chan Zuckerberg Initiative as well as angel investors are helping protocols.io get to a point of self-sustainability.

Bio2040’s take on protocols.io

Needless to say, we are excited about protocols.io and its mission. Here is why we think this is an important initiative

Saving time for researchers

This is a huge bottleneck for researchers starting new projects right now. They will spend months searching through the literature for the right protocol, only to then find out that key steps are missing or aren’t working like they are supposed to. Protocols.io elegantly addresses this problem by providing community reviewed, step-by-step instructions for important scientific recipes.

Faster & open dissemination of protocols

Today, great protocols either never get shared, or they get shared years after they’ve been discovered in the method section of a paper. Not every great protocol makes it into a paper, and when it does, the paper submission maybe years after the protocol has been established. Plus the month-long peer-review process may further delay the publication. Finally, if published in a non open-access journal, only researchers at elite institutions even have easy access to it. Protocols.io solves all of these problems, as long as researchers are willing to take the time and make their protocols public. This immediate publication has the potential to dramatically increase the speed at which the biomedical community advances as a whole.

More meritocratic way to gain reputation

Like github, we see protocols.io as a quite meritocratic platform, where good work gets rewarded through being run in the lab, being bookmarked and forked. Thus, researchers have a new way of gaining reputation which doesn’t depend on the opinions of a few editors and peer-reviewers in scientific journals. Rather, the reputation is gained by a wider community of scientists who use their protocols in the field and can attest that it works as described. This feels like a much more transparent way to gain and assess reputation than the current way it is done.

Thank you Anna Kornakiewicz for helping with writing

Liked this article?
Please help spread the word and share on your favorite social media or send it a good friend who deserves to learn

About Bio2040

There are so many challenges in drug discovery. We are a group of entrepreneurs and scientists who want to improve things. Our first measure is to educate ourselves and the community on what the biggest bottlenecks and their underlying reasons are. This leads us to discover exciting new opportunities. Bio2040 wants to be the leading place for or entrepreneurs & academics to meet, exchange ideas and launch new ventures.

How to stay connected or get involved

Subscribe to the Podcast on iTunes

Be in the know and sign up for the newsletter.

And you should follow Flavio on Twitter here.

If you are working on technology or related research, have something to talk about or would like to contribute, please reach out.