Reading medical literature can be a daunting task, even for people with medical backgrounds. The jargon alone is enough to make your head spin. Combine that paywalls, unique formatting, and complicated statistical methods and it becomes easy to understand why most people simply read someone else’s explanation of what a study says. The problem is that many of the people providing these explanations are biased and may not be well versed in complex statistical methods and the design of clinical studies. That is why, whenever possible, I encourage people to read the study for themselves. But where do you start? First, I’ll discuss the different types of journals and how they relate to the quality of evidence contained in these sources; then we’ll do a quick primer on the format of medical literature.
Does the Journal Matter?
In a word, yes. In medicine, the journal that an article is published in will tell you a lot about the perceived quality and importance of the data being reported. Certain well-respected journals will only publish high-quality research. The New England Journal of Medicine, JAMA, Pediatrics, and Annals of Emergency Medicine are a few of these high-quality journals. Each of these performs a rigorous peer-review process to ensure that the data is reliable, the topic is important, and the conclusions are reasonable.
Below this top tier is a multitude of specialty journals that are also high quality but focus on specific topics such as trauma surgery, cardiology, or pathology. As a person outside a specialty, it can be difficult to judge the quality of specialty journals, so a metric called the “impact factor” was developed. This gives an idea of how often a journal’s articles are cited in other journals, books, and papers. It’s a blunt tool that can be gamed but it’s all most people have outside of asking a specialist what journal they trust.
The bottom tier of peer-reviewed journals is the multitude of pay-to-publish journals that exist. These journals still perform peer review, but the standards are much lower. You can find out if a journal charges a fee to publish by looking at the “information for authors” page. This is also a blunt tool as there are some open access journals (journals that don’t charge a subscription fee and allow free access) of high quality that charge a fee to publish so that the articles are free to view by anyone.
Below the peer-reviewed journals are the pre-publication sites. These became particularly popular during COVID as a means to disseminate information rapidly and widely. This was important because very little was known about how to take care of COVID patients so early access to research was imperative. However, it is important to take studies uploaded to these pre-print sites with a grain of salt. Because they have not gone through peer review, the claims made have not been scrutinized and the methods have not been verified. Be cautious of overly relying on information from these sites.
The final source I’ll discuss is the popular medical literature. This includes newsletters, websites, science periodicals, etc. These are on par with the prepublication sites in the sense that they have not undergone peer review. They are also subject to the bias of the editor who may have an ideological bent. Always be cautious when reading claims from these sources and if possible, look up the citations for any claims that are made.
Medical Journal Article Format
Most medical research articles follow a basic format. Depending on the journal, they may have slightly different names for the sections, but the idea remains the same. First, you will see the abstract. This is a rapid distillation of the paper as a whole down to around 500 words. The abstract should contain the main study results and how the authors of the study interpret those findings. After the abstract comes the introduction. This is where the authors lay out the background and reasons why their study needed to be done. The references in this section are often a gold mine of background information. At the end of this section, the authors will typically state either their hypothesis or the purpose of the study. If this statement is missing, it makes me wonder if the authors had a specific goal in mind when performing their study.
Now we enter the meat of the paper. First comes the methods section. This will describe the type of study, where it was performed, and who was included (more on different types of studies in a different post). For drug trials, you want to pay particular attention to which people were included and which weren’t (inclusion/exclusion criteria). They will often create an ideal situation that has very little resemblance to the real world. After describing the setting and population to be studied, the authors will tell you their primary endpoint (the result they are most interested in). They may also describe some secondary aspects they will look at, but the primary endpoint is what should answer the question/hypothesis stated at the end of the introduction section.
The methods section is also where the authors will describe the statistical methods they used. This section will often make your eyes glaze over, but it is important not to skip it. If you don’t understand the stats, it is difficult to evaluate the validity of the results. Wikipedia has good pages (with examples) for most statistical methods that can give you enough knowledge to move forward. One additional part of the methods that may be described is a power analysis. This is a statistical test that tells you how many people are needed in the study to answer the primary question. All randomized drug trials should have a power analysis.
Next, you finally get to the results. In this section, they will tell you about the people that were included (usually demographics are put in Table 1) along with the results of the study. They may also show a flow diagram of what patients were excluded from the study and at what times that occurred. In this section, the most important findings will typically be put into the text of the article, while other outcomes will be put into tables. There is a trend to move away from using p-values (a value that describes the likelihood that a result would happen by random chance) and to instead look at confidence intervals (the expected range of results if the findings are not due to chance). Overlapping confidence intervals usually mean that there is a decent chance there is no difference between the groups (or no benefit from the drug).
The penultimate section is the discussion section. This is where the authors get to tell you what they think their results mean. Watch out for overly optimistic interpretations of results or overbroad generalizations. This can be a sign of bias on the part of the authors. There should be a limitations section where they tell you how the study could have made errors or may not apply to certain groups.
The final section is the conclusion. In this section, the authors boil down the introduction, results, and discussion into 1 or 2 paragraphs. You will often also see suggestions for future research that can clarify the issue the study seeks to address further. If the authors have potential conflicts of interest (such as funding coming from a pharmaceutical company or being a paid advisor) it is typically placed following this section.
At the end of the paper will be a list of references used in the paper. If you are doing a deep dive into a study, you may need to use this section to review other related papers. If you run across a statement that seems at odds with others, it can be useful to at least review the cited work.
Summary
Which journal an article is in matters. High-quality journals publish high-quality studies. It’s not the only thing, but it is important.
Read the abstract, it will let you know the basics and set you up for reading the rest of the paper.
Don’t skip the methods section. This is where you can figure out how reliable and generalizable the results are.
Make sure that the results match the conclusions in the discussion section. Authors may reveal pre-existing biases in this section by overstating their results or attempting to explain them away. If the author reveals a large bias, it makes me less confident in their results.
Trust yet verify. Go to the cited papers and be sure they are being accurately portrayed. This is particularly important if the study may change your medical care.
What Comes Next
Now that you have a basic understanding of the format of medical literature, we can move on to reviewing an actual article. Look for my Journal Club post later this week. I will be picking an article that is not behind a paywall and will go section by section with my interpretation. However, don’t take my word for it, read along and make your own decision. As always, if you have an article or a topic you would like me to cover, let me know.