Watch the recording from the webinar and download the slides below.
Due to continued interest in the topic, a spin-off project was created following the presentation. Learn more about Project AI Grand Challenges on the project page.
SUMMARY
Unequivocal use of Machine Learning necessitates quality standards and performance metrics of image-derived biomarkers/metrics before introduction in a clinical trial setting and/or daily practice setting can be considered. The level of uncertainty on the added value of machine learning methods for daily practice and trials is critical and need to be mitigated. Tumor Infiltrating Lymphocytes (TILs) can serve as an exemplar application as they represent a valuable predictive and prognostic biomarker in breast cancer, and AI/ML grand challenges provide an opportunity to explore the technological workflows, data formats, and performance assessment methods. This presentation will touch upon the following key points. First, an introduction to the concept of grand challenge in medical imaging, their motivation and some key results. Second, an overview on the grand-challenge.org platform, home to grand challenges in medical imaging with >69,000 active users, as a platform for secure development and validation of AI algorithms. Third, we will introduce two challenge-based initiatives within the context of tumor-infiltrating lymphoyctes (TILs) assessment in breast cancer: CATALINA (https://www.tilsinbreastcancer.org/tils-grand-challenge/), where two existing AI algorithms for TILs assessment are being validated on a set of 7 phase-3 clinical trials with TNBC breast cancer cases; TIGER (https://tiger.grand-challenge.org/), an ongoing public challenge on automated assessment of TILs in TNBC and Her2+ breast cancer. For both projects, we will introduce the motivation, the study design, and the planned analysis. Ultimately, leadership and guidance to the pathology community is needed for the adoption of digital pathology for quantitative biomarkers assessment to improve the standard of care in cancer care. A risk-management based framework, based on the recommendations developed by groups such as The International Immuno-Oncology Biomarker Working Group (www.tilsinbreastcancer.org) and the Pathology Innovation Collaborative Community (PIcc Alliance: www.digitalpathologyalliance.org), will be proposed that can be used by research, trial groups, scientific journals, regulatory agencies, etc. in the evaluation of the analytical and clinical validity of the Machine Learning-tool for the intended purpose. Finally, the presented framework can help in the analytical and clinical validity of machine learning-based assessment of additional biomarkers of importance to breast cancer, like mitotic count or Ki67, beyond the TILs, and it will be argued that this can inform regulatory science.
Featured Speakers
Francesco Ciompi is Assistant Professor in Computational Pathology. He received the Master's degree in Electronic Engineering from the University of Pisa in July 2006 and the Master's degree in Computer Vision and Artificial Intelligence from the Autonomous University of Barcelona in September 2008. In July 2012 he obtained the PhD (cum laude) in Applied Mathematics and Analysis at the University of Barcelona, with a thesis on "Multi-Class Learning for Vessel Characterization in Intravascular Ultrasound". In February 2013 he joined the Autonomous University of Barcelona as postdoctoral researcher, working on machine learning for computer vision and large scale image classification and retrieval. From September 2007 to September 2013 he was also member of the Computer Vision Center. From 2013 to 2015, he worked as a postdoctoral researcher on automated Chest CT image analysis for efficient lung cancer screening at the Diagnostic Image Analysis Group of Radboud University Medical Center. In 2015, he joined the Computational Pathology group of Radboud University Medical Center, working on Deep Learning for automatic analysis of digital pathology whole-slide images.
Roberto Salgado is board certified in Anatomical Pathology since 2006 and works as an Anatomic Pathologist in Antwerp, Belgium. He’s Honorary Research Associate at the Division of Research at the Peter Mac Callum Cancer Center, Melbourne, Australia, as well as being part of the Tracer-X Consortium aiming at deciphering the clonal evolutional pressures in lung and renal cancer (http://tracerx.co.uk/). Currently, he’s co-chairing with Sherene Loi, Peter Mac Callum Cancer Center, Melbourne, Australia, and Carsten Denkert, University of Marburg, Germany, an International Consortium of Pathologists, namely the “International Immuno-Oncology Biomarkers Working Group” (www.tilsinbreastcancer.org) that develops guidelines for the assessment of Immuno-Oncological Biomarkers in cancer. This Working Group is a pathologist-driven group, with >600 pathologists on board, from 47 countries. This Working Group aims to develop guidelines and tools to help pathologists implement TILs in daily and clinical trial practices, publishing >10 manuscripts over the past 5 years. His strategic views on Oncology have been published in major international high impact factor journals such as Nature Reviews Clinical Oncology, Nature Reviews Drug Discovery, and Lancet Oncology. Research work has been published in Nature, The Lancet Oncology, Journal of Clinical Oncology, Nature Medicine. Finally, he’s member of the WHO Editorial Board of the WHO Classification of Breast Tumours, 5th edition.