NIEHS scientists Brandy Beverly, Ph.D., and Andrew Rooney, Ph.D., coauthored the World Health Organization’s (WHO) “Framework for the use of systematic review in chemical risk assessment,” published Nov. 3. The document provides criteria for evaluating and addressing chemical risk assessment research questions, and it was developed in partnership with members of the WHO Chemical Risk Assessment Network (see sidebar).
Rigorous and informative
“Systematic review is a methodology for identifying, selecting, appraising, and synthesizing available evidence to answer a research question, in a way that maximizes transparency and minimizes bias and error,” the authors stated.
Although systematic review approaches were initially used in clinical medicine and social sciences, there is growing interest in applying them in environmental health sciences research.
“Such review is rigorous, stepwise, and instructive,” said Beverly, from the NIEHS Division of the National Toxicology Program (DNTP). “It documents every step of the analysis so that it is easy to see how and why conclusions were reached.”
According to the authors, systematic review methods are particularly helpful in situations where there is conflicting evidence and significant uncertainty — as is often the case when determining the potential health effects of a given environmental exposure.
Bringing clarity to international community
The WHO framework aims to provide institutions with clarity about systematic review principles because the approach is complex, resource-intensive, and time-consuming.
For example, the methodology first involves development of a problem statement that clearly defines the ultimate goals of the assessment and how much analysis is required to provide adequate information about potential health effects of an exposure. Among other efforts, scientists must analyze relevant individual studies, which often are numerous and varied; synthesize the totality of evidence; ensure reliability of results; and draw conclusions.
Rooney, also from DNTP, noted that one challenge for risk assessors involves lack of common systematic review approaches. Difficulty can arise when attempting to synthesize data collected by multiple agencies and countries, which might have different reasons for conducting a review.
“Decisions are very much context-dependent in the framework of individual governments and agencies,” he said. “Our new guidance will help to provide greater understanding and opportunities for harmonization.”
Increased harmonization — combined with ever-advancing computing tools — will bolster systematic review approaches and lead to more timely decision-making, according to Rooney.
Avoiding cherry picking and bias
Through rigorous systematic review, it becomes easier to understand the available evidence regarding potential health effects from an exposure, whether all of the evidence aligns or not.
“For example, there may be one study, a dozen studies, or an entire group of literature on a given chemical exposure, and some of the studies may report potential health effects,” said Rooney.
“To do a useful risk assessment, we use systematic review methodology to clearly present the question being investigated, outline how literature was searched, show how rigorously data were analyzed, and present evidence to support a decision,” he explained. “The approach helps scientists avoid the bias of cherry picking or selecting only the evidence that supports their conclusions.”
Beverly said that she hopes that the WHO publication will provide better understanding of how the systematic review process can be used to assess the health effects of chemical exposures.
“We don't want systematic review to be viewed as an impossible task, and this framework will outline the major steps and considerations for conducting a review,” she said. “Greater understanding of this methodology will hopefully promote greater uptake of systematic review to support decision-making across the international community.”
(Kelley Christensen is a contract editor and writer for the NIEHS Office of Communications and Public Liaison.)