The Alliance held a meeting on October 5th in Orlando, FL –
Hosted by the DPA the Saturday before the Pathology Visions meeting
Before the main meeting (1-5PM) the steering committee had a brief pre-meeting and discussed:
The importance of a Program Manager (1FTE)
Scoring criteria for the proposal and selection of workstreams
Importance of next meeting (MDIC 4th of November)
Proposal for a 2020 Alliance meeting (possibly hosted by the FDA).
The aim of the meeting was to:
Have key stakeholder presentations to inform the members
Share and discuss the aims of this temporary alliance/initiative
Derive 8 concise regulatory sciences deliverables in pre-selected content areas via a breakout session
Enable networking among the members/participants.
Share the content in the public domain
Please find the presentations and results of the breakout sessions below.
Main Talks
THE ALLIANCE – A WELCOME ADDRESS
Esther Abels, PathAI/DPA
Esther set the stage with a highly motivating statement: “Don’t say, just show. Don’t promise, just prove”. She provided the overall scope and aim of the Alliance Visions meeting.
THE ALLIANCE – ALLIANCE VISIONS IN ORLANDO
Jochen Lennerz, MGH
Joe provided an overview of the Alliance for new members, explained the focus on regulatory sciences and the importance of accounting for innovation and the (largely ignored) patient perspectives. The talk also entails the (unpublished) results of the Alliance survey as well as a roadmap of current accomplishments.
MDIC OVERVIEW
Jithesh Veetil, MDIC
Jithesh outlined the role, scope and function of MDIC with a focus on advancing regulatory science of medical devices and diagnostic for patient benefit in the pre-competitive phase. Jithesh’s slides also include the MDIC program structure with concrete initiatives and program areas.
MDDT PROGRAM AND MOCK SUBMISSIONS
Cheng Cui, FDA
The Alliance was excited to have a representative from the FDA present two of the new submission programs. Cheng Cui is a scientific reviewer in the Division of Molecular Genetics and Pathology and oversees the regulation of digital pathology medical devices. He explained the MDDT program including the qualification process. Furthermore, he outlined mock Q-submissions on the example of a Virtual Patient Model. The slide-deck contains several links to highly relevant guidance documents and additional resources. Please take a look.
WHAT TO EXPECT FOR THE FUTURE OF REGULATION AND STANDARDIZATION OF DIGITAL PATHOLOGY
Esther Abels, PathAI/DPA
Esther briefly introduced the role and aims of the DPA as well as the role of the regulatory and standards taskforce. The regulatory goals for 2019 as well as the FDA DPA informational meeting was outlined. Please take a look at the key questions listed on slide 8.
PATIENT INVOLVEMENT IN THE ALLIANCE FOR DIGITAL PATHOLOGY
Laura Lasiter, Friends of Cancer Research
The Alliance was very excited to have Dr. Laura Lasister, the Science Policy Analyst from the patient advocacy group “Friends of Cancer Research” present. Laura explained the importance of patient communities in the medical product development lifecycle and how to promote research through legislation. She outlined the scope and relevance of the Friends of Cancer Research activities and how Friends are accelerating the pace of innovation via policy and advocacy. Please take a look at the very comprehensive slide-deck and also check out their website: https://www.focr.org/
Breakout Sessions
BO / ONE SLIDE SUMMARY
The Alliance Visions meeting entailed a break-out session with 8 focus groups working on pre-defined problems. This slide provides an overview of the 8 topics, the session leaders, and the key deliverables. For each break-out group we provide summary slides below. The Alliance would like to thank all participating members for their contributions and we provide these slides as a resource.
BO-A / PRE-ANALYTICS
The group proposed to develop a guideline and tools to enable generalizability of AI/ML applications and minimize the variability of human factors in this workstream. One important take-home message was the recognized importance of pre-analytical factors clashing with the lack of standardization.
BO-B / SLIDE SCANNING
The group discussed the need for definitions and an educational campaign. Clearly delineated deliverables for analytical studies will save time and expense for device manufacturers and users. Simply put, standardization of slide scanning goes beyond technical performance assessment.
BO-C / TRUTHING DATA SETS
This breakout planned to establish and disseminate a framework for creating truthing datasets with a demonstrated use case to unify human elements of ground truthing. One ambitious goal could be to co-develop a guidance document by subject-matter experts that could serve as the foundation for larger-scale regulatory efforts.
BO-D / ML-CONTINUOUS LEARNING
The group discussed and sought a better understanding of how to verify and validate continuous learning algorithms. Algorithm Change Protocols -as currently described- will hardly capture all ramifications of adaptive algorithms in clinical practice. The group emphasizes the fundamental importance of interoperable systems when it comes to adaptive algorithms.
BO-E / STANDARDIZATION
The group proposed the generation of a database of standards as well as the importance of datasets as a resource and tools for algorithm development. The proposed resources could function as an incubator for method development and alleviate issues related to systems communication, compatibility, and future utility across systems.
BO-F / PRACTICAL USE CASES
A solid business case for the return on investment (ROI) in digital pathology is urgently needed. The group proposed the creation of an ROI calculator to capture data across multiple laboratories. The proposed solutions may streamline creation and review of submissions and ease the burden to industry and FDA in making informed decision.
BO-G / ML/AI-MODEL CREATION
The current guidance on model creation is a monolith that groups all algorithms and purposes together. The group emphasized that AI categorizations is urgently needed. The presentation entailed focusing on addressing different use-cases and purposes for which algorithms are developed. Tailored/categorized evaluation checklists and development tools are concrete deliverables with direct relevance for regulatory sciences.
BO-H / PAYOR STRATEGIES
This breakout session tackled the need for understanding coverage determinations and incorporation of digital pathology into existing clinical guidance and decision-making documents. The group proposed to also include payors into the Alliance discussions and approach clinical guideline developers to ultimately understand how to integrate digital pathology in a financially sustainable and meaning manner.