An Aug. 24-25 symposium on the quickly developing field of systematic review was a highly successful interdisciplinary meeting, according to lead organizer Andrew Rooney, Ph.D., from the National Toxicology Program (NTP). Rooney is acting director of the NTP Office of Health Assessment and Translation (OHAT).
The Fourth International Symposium on Systematic Review and Meta-Analysis of Laboratory Animal Studies drew experts to NIEHS from Europe, Egypt, India, Australia, and North and South America. A full-day training session Aug. 23 offered hands-on experience for individuals new to the field.
Systematic reviews have been used for years in clinical medicine to compare treatment options. Their use in toxicology and environmental health sciences is relatively recent, and methodologies are developing rapidly.
“The goal of the symposium was to increase awareness of the utility of this approach, the transparency the method offers, and share updates on methods,” explained OHAT Health Scientist Vickie Walker, another key organizer.
Powerful, transparent basis for decisions
“Systematic reviews are a powerful tool because they allow us to see more clearly the state of the evidence, and move faster to decision making,” said keynote speaker Tracey Woodruff, Ph.D., from the University of California at San Francisco (UCSF).
A systematic review helps answer a specific research question by collecting and analyzing data from published studies, using clearly specified approaches (see sidebar). Meta-analyses use statistical methods to combine data from multiple studies.
Researchers establish criteria in advance, such as search strategies, criteria for including or excluding studies, and quality assessment that they will use to identify, select, and critically assess the evidence. “With systematic reviews, each step is transparently reported,” Walker explained.
This transparency makes the results useful to decision makers and enhances reproducibility — a priority at the National Institutes of Health and other research institutions. “Systematic review methods facilitate reproducibility because of the transparency in both methods and scientific judgments required to reach evidence-based conclusions,” said Walker.
Challenges in environmental health applications
Public health applications of systematic literature review and meta-analysis of laboratory animal studies face numerous challenges. For example, data heterogeneity is greater than that in clinical research. Environmental health studies may vary widely in key features such as exposure timing, dose, route of administration, model species and sex studied, and endpoints measured to assess any given health effect.
“While we love these things [systematic reviews], they are incredibly laborious to produce,” said keynote speaker Byron Wallace, Ph.D., from Northeastern University. A new tool he is developing, RobotReviewer, aims to reduce the human work by half, through machine learning and text mining. “The technologies for doing this are relatively mature," Wallace pointed out.
Another keynote speaker, Kim Wever, Ph.D., focused on meta-analysis of toxicity data from laboratory animal studies. Wever is a postdoctoral researcher with the Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE) at the Radboud University Medical Center in The Netherlands.
Clinical studies may involve dozens to hundreds of individuals, whereas toxicity studies may involve only a handful. Consequently, statistical methods developed for clinical research may produce misleading results when applied to animal studies.
Carol Kwiatkowski, Ph.D., from The Endocrine Disrupter Exchange, turned to a challenge that arises after the study is complete — publication. Her experience suggests that the field’s relative youth means that many reviewers do not understand the methodologies used.
“Choose your journal wisely,” she advised. “Choose a journal familiar with systematic reviews and statistical analyses.”
Training holds the key
Organizers of the symposium advanced researcher and reviewer knowledge through a training session, led by scientists from SYRCLE, the UCSF Navigation Guide, and NTP. Trainers covered the key steps in performing a systematic review and provided hands-on exercises.
Stacey Mantooth, a contractor with the NIEHS Library, valued the experience. “Demonstrations of the tools and expertise needed to progress through each step of the review showed me what role an information specialist might take in the process,” she said.
Symposium organizers reflected worldwide involvement in the field.
- Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES).
- The U.S. Environmental Protection Agency (EPA) National Center for Environmental Assessment and Integrated Risk Information System (IRIS)
- The University of Tasmania School of Medicine.
- The UCSF Navigation Guide.
- The Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE).