Cochrane crowd

Frequently asked questions about Cochrane Crowd

What is Cochrane?

Cochrane is an international organisation that summarises the best evidence from research to help people make informed choices about treatments.

The need for Cochrane's work is even greater than it was when we started 20 years ago. As access to health evidence increases, so do the risks of misinterpreting complex content; meanwhile the likelihood of any one person getting a complete and balanced picture decreases. Our mission to provide accessible, credible information to support informed decision-making has never been more important or useful for improving global health.

You can find out more about the organisation here: http://www.cochrane.org/about-us

What is Cochrane Crowd?

Cochrane Crowd is Cochrane’s citizen science platform. It was launched in 2016. It hosts small discrete tasks that help identify and describe health research. Anyone is welcome to sign up and join this collective effort. Accompanying each task is a comprehensive, yet brief, training module. Watch our 2-minute animation about Cochrane Crowd to learn a little more about what the platform is and how it works.

What tasks are available on Cochrane Crowd?

We have a number of tasks on Cochrane Crowd. We call the tasks ‘micro-tasks’. When you first start using Cochrane Crowd, you will only be able to see certain micro-tasks. As you progress within Cochrane Crowd, you will ‘unlock’ new micro-tasks!.

The main micro-tasks we have are about helping us to identify certain types of health research: randomised trials.

How is the work I do in Cochrane Crowd of benefit to Cochrane?

The work that our contributors do in Cochrane Crowd is incredibly useful to Cochrane in a number of ways. For example, with the randomised trial identification task, we need these citations screened. They feed a unique and very important product to the world of evidence based medicine: the Cochrane Central Register of Controlled Trials, CENTRAL. This product helps us to find trials quickly and easily which in turn helps to answer vitally important questions about the effects of a treatment. The more up-to-date and comprehensive CENTRAL is, the more effective we can be in answering those questions quickly and accurately. Considered as a task on its own, it can be difficult to see just how important it is, but it is fundamental to what we do. Without identifying the trials, we cannot appraise them, assess them, meta-analyse them; we cannot accurately answer those questions about the effectiveness of new treatments or interventions.

To find out more about CENTRAL and how it is created go to: http://www.cochranelibrary.com/about/central-landing-page.html

To find out more about national access to the Cochrane Library go to: http://www.cochranelibrary.com/help/access-options-for-cochrane-library.html

Where do the records come from that the community work on?

Records come from a range of sources, some freely accessible such as from ClinicalTrials.gov or PubMed. However some of our sources, now and in the future, are and will be from proprietary databases. At the moment we are working on records that have been sourced from Embase (via the Ovid SP platform). Embase is owned by Elsevier, a for-profit publisher with whom our publisher, Wiley has a contract. This enables us to access and re-publish those records in CENTRAL. Wherever possible we seek to share the records identified by the community with as wide an audience as possible, within a framework that acknowledges and respects the legal rights and efforts of database providers.

Is the Crowd generated data open access?

Yes, any meta-data that the Crowd creates is open. Get in touch with us if you are interested in obtaining a data set for research or non-commercial purposes: crowd@cochrane.org.

What if I make a mistake?

You are bound to make a few mistakes, especially when you first start doing this. Even experienced screeners misclassify the odd record. We have created a very robust agreement algorithm to help counter this so don’t worry; no single citation is screened by only one person and disagreements are arbitrated by expert screeners.

Can I use Cochrane Crowd for my students?

Yes, absolutely! We have built a tool called Classmate to help trainers, teachers and others use the Cochrane Crowd micro-tasks for their students. You can access Classmate here: crowd.cochrane.org/classmate. It is in beta form and will remain in beta until the end of 2017. If you have any queries or questions about Classmate or about using Cochrane Crowd for your students, please do get in touch: classmate@cochrane.org.

Can I use Cochrane Crowd for my systematic review?

Cochrane review authors will be able to access the Crowd for their review under a new service called Screen For Me. This service is currently being piloted, evaluated and refined, and is due for wider rollout over the coming months. Watch this space!

Can I choose to work on records in particular topic areas?

Yes, you can do this. You need to go to your Settings. There you will see a setting called: Prioritise the records I receive. Enter a word or short phrase that describes what you are interested in e.g. dementia, and we will prioritise records containing those terms for you. If you don’t get records containing any of those terms, it is because we don’t have in the current batch being screened.

Can I promote Cochrane Crowd to people who are interested in certain healthcare conditions or diseases?

Yes. If you want to promote Cochrane Crowd to a group of people who will likely have a particular interest in a certain topic area, such as neonatal health, you can now do that. We have created around 31 topic lists (and are happy to create more!) to choose from. To promote Cochrane Crowd at the topic level you just add the topic name to the end of the Crowd url. Staying with our neonatal example, if you want to let people know they can focus on records likely to be to do with neonatal health, you share this url: http://crowd.cochrane.org/neonatal

If you would like us to create one for you, just get in touch: crowd@cochrane.org

What if I sign up and then realise I don’t really want to do it after all?

This will rip a hole in the space-time continuum and it’ll be hard to put right. Of course, I’m only joking. If you sign up and then find you don’t want to do this after all, or you simply don’t think you have the time, then that is absolutely fine. You don’t need to feel bad or to notify us. We expect that some will sign up and then realise that actually it’s not for them after all.

Can I screen on my tablet or my smart phone?

Yes, the screening tool is web-based but works really well on tablets and on phones

Can I work offline?

Yes. The way Cochrane Crowd works is that a small number of records are downloaded for you automatically. These are then automatically synced with the server. If you know you’ll be offline for a while and want to do this task, then you can select to download up to a 1000 records to work on. This feature is accessible from your Settings..

What do I get out of this?

The Cochrane Crowd micro-tasks are all designed to be doable for anyone with an interest in health. Some will take a bit longer to get the hang of than others, but all are a great way to become more familiar with what is happening in the world of health research.

We’ve designed Cochrane Crowd so that as you cross certain milestones you are rewarded. For each task, you can earn up to five milestone badges!

The badges will appear on your task dashboard as you achieve them! With each task you start off with the training module for that task. Once you complete it, you will earn your green badge. The next three badges, bronze, silver and gold, are awarded as you hit a certain number of classifications made. To achieve any of these badges is fantastic and you can aim to collect gold badges across all the Cochrane Crowd micro-tasks.

To earn your purple badge you’ll need to be truly exceptional at the task. It not expected that many people will reach this level. It requires a minimum of 1000 classifications combined with very high accuracy. Only around 1% of the Cochrane Crowd community has a purple badge.

In addition to earning your Cochrane Crowd badges, you can now also earn Cochrane membership through the work you do on Cochrane Crowd. The Cochrane Membership Scheme was launched in September 2017. This exciting initiative means that anyone can ‘join’ Cochrane, and your journey to greater contribution can start with Cochrane Crowd. 1000 classifications in Cochrane Crowd within a 12-month period will mean that you will be offered Cochrane membership.

Finally, you will also be helping Cochrane in tasks that need doing. Cochrane is an international not-for-profit organisation. Our work is internationally recognised as the benchmark for high quality information about the effectiveness of health care. By helping us, you will become a part of that effort to produce high-quality, relevant, accessible systematic reviews and other synthesised research evidence.

Can I feedback my thoughts to the project team?

We would love to hear your feedback on any aspects of Cochrane Crowd. Please don’t hesitate to contact me: Anna Noel-Storr either at anna.noel-storr@rdm.ox.ac.uk or crowd@cochrane.org