In research
In our first scenario we looked at how the media portrays health research. Now let's look at how to critically think about the world of health research and how we know when to trust it.
Two Letters to the Editor
Here we have taken quotes from two Letters to the Editor from the New England Journal of Medicine. Consider what role these letters have played in Health Research, and public health at large.
Letter to the Editor 1
We conclude that despite widespread use of narcotic drugs in hospitals, the development of addiction is rare in medical patients with no history of addiction.
Letter to the Editor 2
The crisis arose in part because physicians were told that the risk of addiction was low when opioids were prescribed for chronic pain.
A one-paragraph letter that was published in the Journal in 1980 was widely invoked in support of this claim, even though no evidence was provided by the correspondents
Note
For further information check out the full article Addiction Rare in Patients Treated with Narcotics from 1980 and A 1980 Letter on the Risk of Opioid Addiction from 2017.
Role of Letters to the Editor
Letters to the editor are shorter and more focused than traditional articles. Although they may be subject to peer review (often at the editors' discretion), and may be edited for clarity or length, they generally do not have the same requirements or processes as a published full article.
A letter to the editor generally takes one of the following forms:
- A substantial analysis of a previously published article. For example, to constructively point out potential flaws or discrepancies
- A new perspective, such as a case study, a novel adverse drug event or an important issue relevant to clinical practice. The intention of the letter is to be pertinent and timely but the topic does not justify the presentation of a full length paper.
Given the brevity and lack of rigour of Letters to the editor, these should be considered carefully when citing. As these may contain anecdotal evidence or limited data, these should not be used in isolation or without an acknowledgement of their context.
However, when they are well written or in well regarded journals, they can impact on the direction of research, they can question published results and theories, and review previously published articles.
What am I reading!?
- Is it the full story? We might have to read more to find out...
- Are the references independent? Or are they by the same authors?
- What was the author's purpose behind the letter? What's their intention?
- Do letters to the editor have any influence in health research? This letter was cited more than 500 times by researchers and pharmaceutical companies.
- This letter has been published in a well regarded journal, the New England Journal of Medicine. Does this mean I can be assured of its quality?
- Is this more trustworthy than the previous letter? It's not a full study but it shows its work with supplemental information.
- Also published in the New England Journal of Medicine, which has a good reputation for quality.
- What were the authors' intentions in this letter? What motivated these authors to write 37 years later?
- Does this reference original research? The references aren't extensive, but the analysis is in depth.
- Were the references by the same authors as the letter? These authors did not reference themselves, they reference other works which is recommended practice.
What can we learn?
Here are some facts the 2017 letter pointed out:
In the end...
In the end the Journal added this note to the top of the original 1980s letter.
Editor’s Note (added May 31, 2017): For reasons of public health, readers should be aware that this letter has been “heavily and uncritically cited” as evidence that addiction is rare with opioid therapy. Leung et al. describe its history.
Does this resolve the issue?
- Will people notice or read the Editor's Note?
- What would you do?
- Will the original letter still be cited by others?
Here you can read what a typical journals' standards are when it comes to editor's notes, corrections, and retractions.
Activity: What if this happened to you?
As a health professional, you need to consider that this may happen to you. Please reflect on what you can do to avoid this.
Do you feel confident that you could avoid being misled?
Here are some tips and tools we use:
- Retraction watch
- Remember the steps of critical thinking. The key is asking questions!
- Learn to evaluate the quality of a study
- Read When to trust (and not to trust) peer reviewed science
- Read broadly - read the cited articles, find other articles on the topic by other authors
- Check for potential bias, e.g. commercial interests may create a conflict of interest when funding clinical research