FIMM researchers have demonstrated that it is possible to achieve good consistency between laboratories for drug response measurements by paying careful attention to the harmonisation of laboratory assays, and by using similar experimental and computational procedures. The research is funded by the Academy of Finland.
In late 2013, a striking report was published in Nature, in which the authors cross-compared published data sets from the Broad and Sanger institutes in overlapping cancer cell lines. Their conclusion was that while gene expression profiles were consistent, drug sensitivity measurements showed highly discordant results. This study highlighted reproducibility issues in cancer pharmacogenomics, thereby echoing the recent concerns on the general lack of reproducibility of pre-clinical results, but at the same questioned the feasibility of functional profiling-based precision oncology.
Compelled to find out if this really is the case for all drug sensitivity measurements, a group of FIMM researchers started to compare the Sanger and Broad datasets against FIMM drug response data. They confirmed the poor consistency between the Broad and Sanger drug responses, but also observed a good level of consistency between the FIMM and Broad drug response results. The improved consistency originated from similarities in the assays, both experimental and computational procedures, and importantly, from using the same comparison setup.
These new results, published in Nature as Brief Communications Arising article today December 1st, show that it is possible to achieve good inter-laboratory concordance in drug response measurements.
“Even though we have seen very high reproducibility at FIMM in our high-throughput drug response profiling studies, it was important to show that also between-laboratory reproducibility is indeed achievable”, commented Professor Tero Aittokallio from FIMM, the lead author of the study.
The results also indicate that the use of comparable laboratory and computational methods is critical for improved consistency.
“It will be important to develop global standards, similar to the MIAME standard for microarray data, not only for the laboratory assays but also for the computational procedures being used in drug data processing”, continued John Patrick Mpindi, a bioinformatician who was responsible for the data analyses.
The comparative work by FIMM provides a more positive view on the reproducibility of drug response screening compared to some of the recent publications, and hence supports the potential of pharmacogenomics and precision medicine, provided that sufficient standardisation of drug sensitivity assays is achieved.
John Patrick Mpindi, Bhagwan Yadav, Päivi Östling, Prson Gautam, Disha Malani, Astrid Murumägi, Akira Hirasawa, Sara Kangaspeska, Krister Wennerberg, Olli Kallioniemi and Tero Aittokallio. Consistency in drug response profiling. Nature, Brief Communications Arising, doi:10.1038/nature20171