This post was written by John Byrne, the Wikimedian in Residence at Cancer Research UK. It was first published here under a CC-BY-NC-SA licence.
In our increasingly internet-enabled world, answering a question or checking a fact can be just a few clicks, swipes or touches away.
In many cases these searches are likely to leave you looking at a Wikipedia page. And if that burning question relates to your health, the desire for information can be far more pressing.
In the case of any health concern it’s important to see your GP as a first port of call. But as more people turn to the web for information as well, how can you be sure that the articles you’re reading on Wikipedia, for example, are accurate and up to date?
This question reared its head in May as numerous media outlets covered US research published in The Journal of the American Osteopathic Association. The research set out to examine the reliability and accuracy of Wikipedia’s coverage of the “Top 10 most costly conditions in terms of public and private expenditure in the United States”.
These include cancer, and Wikipedia’s page on lung cancer came under scrutiny from the researchers.
While it’s hard to disagree with the overall take-home message of the stories – that people shouldn’t be relying solely on sources like Wikipedia to diagnose themselves (something Wikipedia itself is completely clear on) – the study leaves little room for suggestions on how Wikipedia could be improved for patients and the general public – something Cancer Research UK is actively involved in, as we’ll discuss below.
It also led to headlines claiming that “90% of Wikipedia’s medical entries are inaccurate”. Is this a fair representation of the research, and of Wikipedia?
Errors or ‘discordances’?
As the lead researcher on the new study, Professor Robert Hasty, from Campbell University in North Carolina, US, explained in an interview, the study was prompted by seeing young doctors looking things up on Wikipedia.
The use of the site by medical professionals has been the subject of a fair amount of research (e.g. see the summary on p12/13 of this PDF) though mostly looking to answer questions on how much do they do it (answer: a lot) and should they do it (answer: not as a primary source) rather than why do they do it.
In the new study, each of the “10 most costly conditions” the researchers looked at was matched to a relevant Wikipedia article, which was sent out to two randomly assigned junior doctors tasked with assessing the reliability of the content.
They were asked “to identify every assertion (ie, implication or statement of fact) in the Wikipedia article and to fact-check each assertion against a peer-reviewed source that was published or updated within the past 5 years.”
They found many “discordances” in the content, which they later referred to as “errors” in the conclusions of the research (so, unsurprisingly, this word became the focus of the media coverage).
This led them to conclude that “Health care professionals, trainees, and patients should use caution when using Wikipedia to answer questions regarding patient care” and “physicians and medical students who currently use Wikipedia as a medical reference should be discouraged from doing so”.
This was translated into the headlines we mentioned earlier, flagging the “90% inaccurate” figure.
But the design of the study has come in for unusually heavy criticism at the WikiProject Medicine talk page – where Wikipedia’s regular medical editors talk things over.
For example – to pick one out of many points the editors have discussed – the study doesn’t say where these “errors” are, meaning it’s very hard to check or change the articles.
For a general readership
To quote the physicist Freeman Dyson FRS: “Among my friends and acquaintances, everybody distrusts Wikipedia and everybody uses it…. The information that it contains is totally unreliable and surprisingly accurate.” – a useful distinction when looking at Wikipedia.
Wikipedia – and its volunteer editors – have always made it clear that it does not offer medical advice, let alone represent a substitute for professional advice, nor is it a medical textbook.
The internal style manual for medical articles is emphatic that Wikipedia’s medical content is aimed at a general readership, and cautions against writing directed at either patients or medical professionals, as well as banning the inclusion of information such as pharmaceutical dosages.
In practice, however, many of the articles do contain technical terms, and this can make some of them difficult for the average member of the public to understand. Clearly, there’s room for improvement.
Wikipedian in residence
In 2011, Cancer Research UK approached Wikipedia to see if the two organisations could work together to improve the cancer-related content on the site. This led ultimately to my appointment as the charity’s Wikipedian in Residence. The role will run until mid-December 2014, and is funded by the Wellcome Trust.
Part of my role here will be to work with the existing medical editors on Wikipedia to improve our articles on cancer topics, in particular those on four harder-to-treat cancers where there has been little improvement in survival rates in recent decades. These are cancers of the lung, pancreas, brain and oesophagus, which Cancer Research UK is giving particular focus to as part of its new research strategy.
But I will also be addressing other cancer-related content, for example for the Medical Translation Project that translates articles between the over-200 different language versions of Wikipedia.
Cancer Research UK has access, through its own staff and its access to other researchers and clinicians, to tremendous amounts of expertise, both in terms of science and the communication of science, where they have teams trained and experienced in communicating with a wide range of distinct audiences – including through its flagship patient information content.
I’ll be exploring a number of approaches to bringing all this expertise to bear on Wikipedia’s cancer content. The very large annual nerd-fest conference Wikimania 2014 is in the Barbican in London this year, about a mile from the charity’s HQ. This gives a great opportunity to bring Cancer Research UK and many medical Wikipedians together face to face.
Another aspect of the role is that we are planning to conduct research into the experiences of a range of different types of consumers of Wikipedia’s cancer content. There has been very little formal qualitative research into the experiences of Wikipedia’s readers – we hope this project will begin to address this gap, as well as encouraging others to carry out similar projects.
It’s important that we all work hard to reduce unreliability and make the accuracy less of a surprise in Wikipedia’s cancer articles. If you are curious, or interested in helping in any way, please do get in touch below or on my Wikipedia Talk Page. It would be sad if today’s media reporting put medical professionals off engaging with Wikipedia – the site, and the public, need your help.
John
- John Byrne is Cancer Research UK’s Wikipedian in Residence
- More academic research on medicine on Wikipedia.
Reference
- Hasty R.T., Garbalosa R.C., Barbato V.A., Valdes P.J., Powers D.W., Hernandez E., John J.S., Suciu G., Qureshi F. & Popa-Radu M. & Wikipedia vs Peer-Reviewed Medical Literature for Information About the 10 Most Costly Medical Conditions., The Journal of the American Osteopathic Association, PMID: 24778001