U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.


The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Environmental Factor

Environmental Factor

Your Online Source for NIEHS News

September 2017

Systematic review symposium explores powerful tool

At an interdisciplinary meeting on the quickly developing field of systematic review, scientists from across the globe shared new methods.

Andy Rooney Rooney co-authored an editorial in the journal Environmental Health Perspectives describing the reproducibility offered by systematic reviews. (Photo courtesy of Steve McCaw)

An Aug. 24-25 symposium on the quickly developing field of systematic review was a highly successful interdisciplinary meeting, according to lead organizer Andrew Rooney, Ph.D., from the National Toxicology Program (NTP). Rooney is acting director of the NTP Office of Health Assessment and Translation (OHAT).

The Fourth International Symposium on Systematic Review and Meta-Analysis of Laboratory Animal Studies drew experts to NIEHS from Europe, Egypt, India, Australia, and North and South America. A full-day training session Aug. 23 offered hands-on experience for individuals new to the field.

Systematic reviews have been used for years in clinical medicine to compare treatment options. Their use in toxicology and environmental health sciences is relatively recent, and methodologies are developing rapidly.

“The goal of the symposium was to increase awareness of the utility of this approach, the transparency the method offers, and share updates on methods,” explained OHAT Health Scientist Vickie Walker, another key organizer.

Powerful, transparent basis for decisions

“Systematic reviews are a powerful tool because they allow us to see more clearly the state of the evidence, and move faster to decision making,” said keynote speaker Tracey Woodruff, Ph.D., from the University of California at San Francisco (UCSF).

Tracey Woodruff “[Systematic review] helps us separate the science piece from the values and preferences used in decisionmaking,” said Woodruff. (Photo courtesy of Steve McCaw)

A systematic review helps answer a specific research question by collecting and analyzing data from published studies, using clearly specified approaches (see sidebar). Meta-analyses use statistical methods to combine data from multiple studies.

Researchers establish criteria in advance, such as search strategies, criteria for including or excluding studies, and quality assessment that they will use to identify, select, and critically assess the evidence. “With systematic reviews, each step is transparently reported,” Walker explained.

This transparency makes the results useful to decision makers and enhances reproducibility — a priority at the National Institutes of Health and other research institutions. “Systematic review methods facilitate reproducibility because of the transparency in both methods and scientific judgments required to reach evidence-based conclusions,” said Walker.

Challenges in environmental health applications

Public health applications of systematic literature review and meta-analysis of laboratory animal studies face numerous challenges. For example, data heterogeneity is greater than that in clinical research. Environmental health studies may vary widely in key features such as exposure timing, dose, route of administration, model species and sex studied, and endpoints measured to assess any given health effect.

“While we love these things [systematic reviews], they are incredibly laborious to produce,” said keynote speaker Byron Wallace, Ph.D., from Northeastern University. A new tool he is developing, RobotReviewer, aims to reduce the human work by half, through machine learning and text mining. “The technologies for doing this are relatively mature," Wallace pointed out.

Statistics, publication

Another keynote speaker, Kim Wever, Ph.D., focused on meta-analysis of toxicity data from laboratory animal studies. Wever is a postdoctoral researcher with the Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE) at the Radboud University Medical Center in The Netherlands.

Shapiro, Tsaioun, and Wever Wever, right, described her statistical methods to NTP data scientist Andy Shapiro, and Katya Tsaioun, Ph.D., from Johns Hopkins University. (Photo courtesy of Steve McCaw)

Clinical studies may involve dozens to hundreds of individuals, whereas toxicity studies may involve only a handful. Consequently, statistical methods developed for clinical research may produce misleading results when applied to animal studies.

Carol Kwiatkowski, Ph.D., from The Endocrine Disrupter Exchange, turned to a challenge that arises after the study is complete — publication. Her experience suggests that the field’s relative youth means that many reviewers do not understand the methodologies used.

“Choose your journal wisely,” she advised. “Choose a journal familiar with systematic reviews and statistical analyses.”

Training holds the key

Organizers of the symposium advanced researcher and reviewer knowledge through a training session, led by scientists from SYRCLE, the UCSF Navigation Guide, and NTP. Trainers covered the key steps in performing a systematic review and provided hands-on exercises.

Stacey Mantooth, a contractor with the NIEHS Library, valued the experience. “Demonstrations of the tools and expertise needed to progress through each step of the review showed me what role an information specialist might take in the process,” she said.

Symposium organizers reflected worldwide involvement in the field.

Scientists in the audience Scientists from NIEHS, the local area, and across the world participated in person and via webcast, including from right, Malcolm Macleod, Ph.D., from the University of Edinburgh; Gillian Currie, Ph.D., from CAMARADES; and Emily Sena, Ph.D., from the University of Edinburgh. (Photo courtesy of Steve McCaw)
Walker speaking Walker spoke on “Systematic Evidence Map of Transgenerational Inheritance of Health Effects.” (Photo courtesy of Steve McCaw)
John Bucher and Juleen Lam From right, John Bucher, Ph.D., NTP associate director, listened alongside Juleen Lam, Ph.D., from UCSF, who described proof-of-concept case studies using the UCSF Navigation Guide; and Marc Avey, Ph.D., from ICF, who discussed the PRISMA reporting tool. (Photo courtesy of Steve McCaw)
Rob de Vries During the symposium, Rob de Vries, Ph.D., from SYRCLE, discussed the need for education. He was a lead organizer of the training offered before the meeting. (Photo courtesy of Steve McCaw)
Kris Thayer Kris Thayer, Ph.D., leads the EPA IRIS program, one of the symposium’s sponsors. She is the former head of OHAT. (Photo courtesy of Steve McCaw)
Jeffrey Everitt and de Vries Jeffrey Everitt, D.V.M., from Duke University and de Vries took advantage of the poster session and social hour, sponsored by the Evidence-Based Toxicology Collaboration, for further exchange. (Photo courtesy of Steve McCaw)
Back To Top