skip navigation
Environmental Factor, February 2015

Whole Issue PDF
This issue's PDF is still being created and should be available 3-5 business days after the first of the month. Please check back in a few days.

Reproducibility advanced by new NTP systematic review handbook

By Kelly Lenox

Headshot of Thayer

Thayer oversees NTP evaluations to assess the evidence that environmental substances cause adverse health effects. (Photo courtesy of Steve McCaw)

Rooney standing at a podium

Rooney reviewed how his team used systematic review as part of a study of inflammation and atherosclerosis, in a Dec. 2014 inflammation seminar at NIEHS (see story). (Photo courtesy of Steve McCaw)

The National Toxicology Program (NTP) released a handbook Jan. 12 detailing procedures for systematic review in literature-based health assessments. The NTP Office of Health Assessment and Translation (OHAT), which developed this methods guide for evaluating potential human health hazards, hopes use of the handbook will enhance reproducibility of environmental health sciences research, which is a growing concern across all fields of scientific research and is the subject of recent National Institutes of Health initiatives.

As part of the systematic review process, the “Handbook for Conducting a Literature-Based Health Assessment Using OHAT Approach for Systematic Review and Evidence Integration” features a tool for assessing the risk of bias in a study’s findings. The tool applies an approach to evaluating the risk of bias in human and animal studies that is similar to the methods used to critically assess clinical trials of new drugs.

“Establishing transparent and objective standards for systematic reviews in literature-based evaluations builds confidence in the subsequent public health decisions,” said Kristina Thayer, Ph.D., director of OHAT. “And as the methodology becomes adopted more widely, the reproducibility of environmental health studies will be enhanced.”

From clinical trials to broader application

“Interest has been growing in the fields of toxicology and pharmacology in extending systematic review methods beyond the traditional area of human clinical trials to consider other evidence streams,” said NTP leaders and staff in a July 2014 editorial in the journal Environmental Health Perspectives. Authors include NIEHS and NTP Director Linda Birnbaum, Ph.D., NTP Associate Director John Bucher, Ph.D., Thayer, and others.

“This handbook provides the methods for completing a literature-based evaluation for environmental health questions, and it addresses whether or not exposure to chemical X is associated with health effect Y,” said Andrew Rooney, Ph.D., OHAT deputy director. “This document is intended to outline NTP methods, but it can also serve as a methods SOP [standard operating procedure] for external researchers,” he said, noting that this is the first handbook to cover systematic review methods for environmental health questions.

Risk of bias tool

Reproducibility of scientific studies increases when consistent means are used to assess internal validity, or risk of bias. Bias can lead to either underestimation or overestimation of an observed effect. According to the editorial quoted above, there is a general consensus on methods used to assess outcomes of human clinical trials. However, environmental health sciences research uses studies of widely varying design — from epidemiology to mechanistic studies. “There is not currently a similar consensus on how to assess that the findings and conclusions of observational human, experimental animal, and in vitro studies are a true reflection of the outcome,” the editorial’s authors said.

To meet that need, OHAT developed a risk of bias tool, included in the handbook, for evaluating environmental health studies in humans or animals. Rooney pointed out that, as with the systematic review methodology, the tool outlines NTP methods and may also be used by external researchers to assess study quality.

"NIEHS researchers determine new ..." - previous story Previous story Next story next story - "NIEHS Superfund research training ..."
February 2015 Cover Page

Back to top Back to top