DREAM challenge results published in Nature Biotechnology
By Robin Mackar
An innovative international crowdsourcing challenge launched by NIEHS and partners in 2013 has come full circle with an Aug.10 publication in Nature Biotechnology.
"Prediction of human population responses to toxic compounds by a collaborative competition” takes readers through the Dialogue for Reverse Engineering Assessments and Methods (DREAM) Challenge process and describes the development of computational models to predict potential adverse health effects in populations. “The challenge offered the unique opportunity to compare performance across a wide variety of state-of-the-art methods for the prediction of cytotoxic response to environmental compounds,” the authors wrote. Allen Dearry, Ph.D., and Raymond Tice, Ph.D., from NIEHS were among the approximately 140 authors involved in developing the publication.
Meeting the challenge
Using data generated as part of the Toxicology in the 21st Century program
(Tox21) effort, this community-based toxicogenomics challenge was organized by scientists from NIEHS, the National Center for Advancing Translational Sciences, the Carolina Center for Computational Toxicology at the University of North Carolina at Chapel Hill, DREAM, and Sage Bionetworks.
The organizers provided participants with cellular toxicity data generated by testing 156 different chemical compounds on 884 distinct lines of lymphoblastoid cells, a cell type scientists have found especially useful in molecular and functional studies. The cell lines represented individuals from nine subpopulations in Europe, Africa, Asia, and the Americas. Data on single nucleotide polymorphisms (SNP) were available from the 1000 Genomes Project for all the cell lines. Gene expression data were also available for a smaller number of lines.
The challenge included two subchallenges that allowed participants to take on either or both. The first challenge asked the investigators to use the biological data (SNPs, basal gene expression) provided on one group of cell lines to develop a model that accurately predicted responses of other cell lines to compound exposure. The other challenge involved using intrinsic chemical properties of a subset of chemicals to develop a model that accurately predicted how a particular population would respond to other chemicals.
Representing more than 30 countries, 213 people participated in the challenge. Thirty-four teams responded with 99 submissions to the first subchallenge, and 24 teams responded with 85 submissions to the second. The Quantitative Biomedical Research Center at the University of Texas Southwestern Medical Center in Dallas, Texas won both challenges by providing the most accurate predictions.
“The ability of the top teams to predict population-level toxicity for unknown compounds — based on similarities in chemical structure to known compounds — far surpassed our anticipations,” said Lara Mangravite, Ph.D., director of systems biology at Sage Bionetworks. “This was a true case where crowdsourcing the problem provided answers that would otherwise never have been found.”
Kudos to NIEHS partners
Much of the credit for bringing this challenge to fruition goes to Stephen Friend, M.D., Ph.D., president of Sage Bionetworks and Gustavo Stolovitzky, Ph.D., from the IBM Computational Biology Center.
“Steven, Gustavo, and their teams have a wonderful vision for tackling biomedical problems using innovative approaches,” said Linda Birnbaum, Ph.D., director of NIEHS and the National Toxicology Program (NTP). “NIEHS and NTP are proud to have participated in this first-ever crowdsourced toxicological effort, which has not only brought us new computational models, but new collaborations as well.”
Citation: Eduati F, Mangravite LM, Wang T, Tang H, Bare JC, Huang R, Norman T, Kellen M, Menden MP, Yang J, Zhan X, Zhong R, Xiao G, Xia M, Abdo N, Kosyk O; NIEHS-NCATS-UNC DREAM Toxicogenetics Collaboration. 2015. Prediction of human population responses to toxic compounds by a collaborative competition. Nat Biotechnol;doi:10.1038/nbt.3299 [Online 10 Aug. 2015].
(Robin Mackar is news director in the NIEHS Office of Communications and Public Liaison, and a frequent contributor to the Environmental Factor.)