It is increasingly important for us to become critical readers. It is easy to find information on any scientific topic, but it is often easier to find poor quality information than high quality, accurate information. Is it really a promising cure for cancer? Is the vaccine actually going to work or even come to market? Is this really the next outbreak that threatens us all? How do we make sense of this information? It is not always easy to distinguish the good articles from the bad.
Last semester in my Microbial Wars class, students selected news articles and presented the article along with a discussion of its merits and problems. Over the course of the semester, the students identified several features that helped in critically evaluating the articles. Below are some of the major points they came up with. You can test your critical reading skills on this article on the HPV vaccine or this one on a transmissible H5N1 influenza virus.
1. Source: does the publication tend to publish reliable and accurate articles?
2. Author: does the author of the article have a science background? Have they written other science articles? Since a journalist has to interpret and explain the research, their ability to understand the research and its context are important, otherwise they may simply be repeating back information from press releases or other sources without critical evaluation of the information.
3. Is the original source easy to find? Are there links, or sufficient details to know who did the original research so that if you want to find the original source, you could do it fairly easily. Do the sources or studies that the article cites or links to actually support their statements, and are they reliable?
4. Is the research being described published in a peer-reviewed scholarly source? If its not peer reviewed, beware! Is it based on conference proceedings? Conference presentations and abstracts are very minimally reviewed, and should not be considered equivalent to a published study. Often the data from a conference proceeding hasn’t been seen by most scientists or the journalist, so use great caution when reading about this kind of unpublished information.
5. We talk a lot about articles being sensationalized, but what does that mean? It means presenting information in a way that provokes interest or excitement, at the expense of accuracy. Watch out: this happens a lot, even in the most reputable news sources and with top writers. Is the article making unsubstantiated claims? Consider carefully what the research has shown or what specifically was tested. Are the statements made in the news article accurately reflecting what the research shows, or is it taking it several steps beyond the results of the research? Remember that scientific research tends to move in slow, baby steps, not leaps and bounds.
6. Balance: What makes a balanced article? Should different perspectives be included? Should contrary opinions be presented? If so, how much weight should be given to “dissenting” views? A common problem is presenting two sides of a debate as if both sides are equally supported. This can lead to a false sense of how much debate there really is among those in the field.
7. Motivations: what are the motivations of the people quoted or referenced in the article? Are those individuals likely to have an unbiased perspective on the research? You will sometimes see comments or quotes from researchers not involved in the particular study being discussed; these can often provide a good perspective.
8. Headline: headlines are written to draw readers in, and so are likely to be sensationalized or inaccurate. (They are also often written by somebody other than the author of the article). Does the headline accurately portray what is in the article? If you only remember one key message from the article, it probably shouldn’t be the headline.
Considering these points when reading an article may help in reading it critically. No single point will make or break an article, but this is at least a good place to start.
Thanks to the students of Bio/STS 172 for interesting discussions and developing this list!
Any other suggestions?
Absolutely, there are grey areas. The key I think is looking at the original research. It doesnt really matter who generated the data – if the data is good, that’s what is important. The research paper is not likely to be sensational. When you read about it in the NY Times or in somebody’s blog, though, may well be. The best way to decide is to just read more. See what others are saying and try to get the original paper and read it. If the article discusses a topic that is not specifically about a new finding, but rather like the HPV article linked to above, attempts to summarize a whole body of work, then there is a lot more reading that you would need to do. Sometimes it may not be worth it – if you are just reading something out of curiosity, it may not be so harmful if its a little sensational; just take the statements with a little grain of salt. But if you are basing a major decision on it, like whether or not to vaccinate, its extremely important to do your homework and check as much information as possible.
Is there anyway to put weight into an article that has gray areas as far as the guidelines go? For example, there is an unknown scientist that published reliable findings in a reputable journal, but the findings are a bit sensationalized even with the evidence given being reliable.
i think they are, David
are the comments working?