Withdrawn: Federal Notice 86 FR 4088 FDA Proposal to Make Permanent Regulatory Flexibilities Provided During the COVID-19 Public Health Emergency

Jan 15

The proposed federal notice has been withdrawn. The official notification can be found HERE.

Screen Shot 2021-04-19 at 9.01.08 AM.png — Summary of the decision in powerpoint format by J. Lennerz

Click here to download

Our primary objective remains: that is to provide the best scientific evidence as input for improving the regulatory framework governing digital pathology and ML/AI devices. The information below is for informational purposes.

Links to additional comments:

Several of the project members wrote a correspondence that was accepted at nature medicine

Link to paper here
Download pdf of paper here

February 2021:

Federal Notice 86 FR 4088 FDA Proposal to Make Permanent Regulatory Flexibilities Provided During the COVID-19 Public Health Emergency (link)

by Exempting Certain Medical Devices From Premarket Notification Requirements; Request for Information, Research, Analysis, and Public Comment on Opportunities for Further Science and Evidence-Based Reform of Section 510(k) Program

Statement from the “Federal Notice 86 FR 4088 Workgroup”
(available for download in editable form here)

The federal notice from January 15, 2021 as currently drafted is a proposal to make permanent regulatory flexibilities provided during the COVID-19 public health emergency that exempt certain medical devices from premarket notification requirements. Furthermore, the proposal states that the devices subject to the proposal have been deemed safe and effective by the FDA, and that the Manufacturer and User Facility Device Experience Database (MAUDE) search interface does not list serious harm. The proposal and The Department of Health and Human Services[1] is “soliciting the public's views on whether premarket review should be permanently waived (…)”[2].

DISCLAIMER ABOUT THIS STATEMENT

a) This statement has not been specifically authorized by the FDA

b) This statement reflects the majority consensus and contributions from many non-FDA stakeholders, solicited through a public convening. While many have contributed, it is challenging to encompass all viewpoints. We rooted our statement in fact wherever possible and captured our current discussion in a working group where we solicited input. We welcome additional comments via email: digipathalliance@gmail.com

c) The focus of PIcc is regulatory science not policy (=health policy). Health policy is within the interest of some of the PIcc members but is not addressed within this statement.

d) This statement (as PIcc in general) focuses on regulatory science and specifically on which regulatory science tools can help inform implementation, regulation, and policy. Keep in mind, medical research can be both the basis for defining evidence-based regulatory frameworks and the subject of health policy itself.

e) The federal notice is very broad. The following statement focuses on the regulatory scientific implications directly related to (*digital) pathology devices – not any of the other technologies or devices.

The specific devices that are the focus of this statement (along with FDA class and product code) are:

Digital Pathology Display (Class II; Product code: PZZ)
Digital Pathology Image Viewing And Management Software (Class II; Product code: QKQ)
Whole Slide Imaging System (Class II; Product code: PSY)
Automated Digital Image Manual Interpretation Microscope (Class II; Product code: OEO)

These 3 devices PZZ, QKQ, and PSY are linked directly to 21 CFR § 864.3700:
            Hematology and pathology devices,
            Pathology instrumentation and accessories,
            Whole slide imaging systems.

The device OEO is linked directly to 21 CFR § 864.1860:

            Hematology and pathology devices,
            Subpart B – Biological Stains
            Immunohistochemistry reagents and kits

Two additional product codes might be of general interest: POK, MYN

Note: POK and MYN are radiological (Title 21 part 892 is radiology); however, radiological device regulations may establish a precedent with relevance for PIcc and digital pathology.

OVERVIEW OF THE FEDERAL NOTICE AND SPECIFIC RELEVANCE FOR PATHOLOGY

Intent

The stated basis for the proposal is that the during the period of temporary approval, medical personnel used the medical devices in real life clinical situations. As currently proposed, the changes in the realm of digital pathology that would be made permanent are referring back to CFR21 864.3700[3] that state:

CFR21 excerpt

“Identification. The whole slide imaging system is an automated digital slide creation, viewing, and management system intended as an aid to the pathologist to review and interpret digital images of surgical pathology slides. The system generates digital images that would otherwise be appropriate for manual visualization by conventional light microscopy.” (…)

“(i) The indications for use must specify the tissue specimen that is intended to be used with the whole slide imaging system and the components of the system.” (…)

“(i) The intended use statement must include the information described in paragraph (b)(1)(i) of this section, as applicable, and a statement that reads, "It is the responsibility of a qualified pathologist to employ appropriate procedures and safeguards to assure the validity of the interpretation of images obtained using this device."(…)

We also note that the FDA released an additional guidance document regarding technical performance assessment of digital pathology and whole slide imaging devices[4].

Effectively, these changes propose that whole slide imaging systems, the software for viewing, and interpretation of tissue specimen would not require premarket authorization. The manufacturer would still be responsible to design and validate the product according to quality management system requirements (e.g., 21 CFR 820)[5]. Another regulatory element that remains unchanged is the labeling special control indicating that:
"It is the responsibility of a qualified pathologist to employ appropriate procedures and safeguards to assure the validity of the interpretation of images obtained using this device."[6] (now a direct quote)

SIGNIFICANCE

Increased utilization in areas such as remote sign-out. There are many arguments in favor of the federal notice (e.g., our “remote sign-out comment” (see below) and/or the DPA statements in support of both the emergency and permanent use of remote sign-out) (https://digitalpathologyassociation.org/real-world-data-for-remote-sign-out and https://digitalpathologyassociation.org/covid-19-updates-resources). Pathology remote sign-out during this pandemic is primarily needed to continue providing high quality patient care during the ongoing public health emergency (where social distancing and remote-work-from-home is highly encouraged to contain the spread of the infection) and simultaneously protecting the pathologist workforce. Prior to the public health emergency related to COVID-19, there was uncertainty about the ability to implement remote diagnostics. The exemption (implemented in April 2020) alleviated this uncertainty, and the federal notice would now make these changes permanent.

Risk assessment. The federal notice proposes the removal of premarket barriers to implementation of these devices by relying on evidence from the MAUDE database to suggest that they are lower risk. It is unclear if the passive reporting mechanism employed by the MAUDE database is a reliable source for informing product risk. For example, the few comments available in the MAUDE database (n<10; one entails that the monitor is too bright)[7] may not be fully representative of performance issues encountered by devices in the field. Further, the timeline between the publication of the “enforcement policy” (response to the public health emergency; April 2020; see Table 1 in federal notice[8]) and November 30, 2020 (the final date used for the MAUDE analysis described in the federal notice; see, for example, Table 4.1 in the federal notice)[9] was relatively short to capture any adverse events that may have resulted from a lack of premarket notification requirements.

The regulatory logic underlying the regulations prior to the federal notice entailed premarket review as a class II device. The federal notice now proposes to make these class II devices exempt[10]. A Class I or Class II device that is exempt from 510(k) requirements must still comply with other requirements (known as regulatory controls) unless the device is explicitly exempt from those requirements as indicated in the regulation for that device type[11].

Clarification of accountability and responsibility. CLIA governs clinical practice and implementations, irrespective of whether a device is FDA authorized. As currently drafted, CLIA oversight is unaffected by the new proposal. The proposal to remove premarket review will remove the FDA’s independent product assessment, and individual laboratories and medical directors relying on this assessment may have to compensate.

Specifically, the lack of independent review may lead to challenges in root cause analyses for device vs. laboratory issues. From a manufacturer’s perspective, the federal notice does not remove quality-system obligations (e.g., design controls, change management, adverse event reporting, postmarket surveillance, and recall authority, etc.). From a CLIA perspective, laboratories would have to evaluate how a lack of premarket notification affects their local implementation.

Standardization and Interoperability.

Premarket review currently does not incentivize larger-scale interoperability in the digital pathology ecosystem. However, interoperability is one of the key demands in the 21st Century Cures Act[12]. Notably, how to achieve interoperability is currently not addressed by premarket review, the 21st Century Cures act, or the currently proposed federal notice. For example, the demand in the market determines the preferred file-format as one component in the interoperability puzzle.

It remains to be determined whether exempting premarket review requirements on individual components of the digital pathology workflow will have an impact on standardization and interoperability between the interacting components of the digital pathology workflow. The federal notice also touches upon the difficult topic of regulatory oversight of the pixel pathway and whether components in end-to-end solutions (WSI devices) can now immediately become interoperable multi-vendor solutions.

Screen+Shot+2021-02-10+at+7.01.59+PM.png

Explanatory note: Under PSY code (https://www.law.cornell.edu/cfr/text/21/864.3700) and the FDA guidance for “Technical Performance Assessment of Digital Pathology Whole Slide Imaging Devices”[13], a whole slide imaging (WSI) device can be grouped in two subsystems: image acquisition and image display (i.e., scanner and workstation). The workstation is composed of the viewing system and the display. In other words, the device has multiple components that form (in regulatory terms) subsystems. Interoperability requires a standardized information exchange as well as system (or device)- level assessments for comparison in performance (e.g., “substantial equivalence”). The federal notice does not comment on this specific aspect (i.e., performance assessment of individual components or subsystems) and this is only a very brief outline of the problem. Several contributors felt strongly about cursorily mentioning this and we defer to the primary guidance document for details (here).

WHETHER THE FEDERAL NOTICE IS ADOPTED OR NOT, REGULATORY SCIENCE DELIVERS IMPORTANT EVIDENCE TO INFORM IMPLEMENTATION AND REGULATORY POLICIES.

What specific data do we need to generate now?

What data can inform adequate and balanced regulation?

Key regulatory science questions:

1) How to adequately capture adverse events and assess risk of the various digital pathology devices?

The federal notice relies on the MAUDE database[14]. Briefly, there are four main databases which have medical device reports of adverse events:

a. the Manufacturer and User Facility Device Experience (MAUDE) database,

b. the Medical Device Reporting (MDR) database,

c. the Medical Device Recalls database, and

d. the Total Product Life Cycle (TPLC) database.

Screen+Shot+2021-02-10+at+7.02.56+PM.png

While integrated into the existing infrastructure (via mandatory reporting by device manufacturers and voluntary reporting by end users), there may not be an optimal regulatory science tool to truly capture and classify the clinically relevant adverse events or risks of the various digital pathology devices. The resulting two regulatory science questions are:

A) Are we adequately capturing adverse events? Specifically, we want to understand if the data in the MAUDE database adequately captures the risks encountered in clinical practice. One regulatory science approach to assessing the MAUDE database could be a 360o assessment the content and connectivity of the existing databases. Systematic identification of other relevant sources of information and connectivity would also be informative. A collaboration with experts within and outside the FDA could assess the status quo with a specific focus on best ways to make use of the existing database infrastructure. This could create either assurance for adequate capture or identify areas for possible improvements.

B) Can we create a classification system for adverse events in digital pathology? There is no regulatory-grade way to adequately classify adverse events specifically related to digital pathology and AI in diagnostics for laboratories. The existing medical device-reporting[15] or postmarket programs[16] provide the necessary infrastructure by which to collect reports. Utilization will avoid creating redundancy; however, there is no domain specific dictionary to capture and specify the various types and degrees of events (e.g., diagnostic error vs. clinical consequences). Furthermore, there is no systematic way to account for severity (e.g., minor, major, severe, death), which will require careful design and testing – ideally in collaboration with the regulator. While complexity categorizations for in vitro diagnostic devices are available[17], the field is now tasked to develop a dictionary that enables appropriate capture of the varying types and degrees of adverse events. Organization and aggregation are possible through a classification system that granularly captures adverse events.

2) How to stratify risk of the different components in the digital pathology workflow and in particular the pixel-pathway?

Histopathological diagnosis is a multi-modal workflow with many interconnected devices. Digital pathology is a multi-part component in a more complex workflow[18]. Individual components and their relative contribution to the overall performance are very hard to assess. For example, the quality of the monitor and its influence on overall performance might represent a minor component of the overall performance of a diagnostic pathway when compared to the scanner producing out-of-focus images and/or the tissue being lost. However, we do not know the relative contribution of each component. We need a scientifically sound and regulatory compliant approach to assess the influence and risk of individual components on the overall diagnostic performance. The total inaccuracy and imprecision of a digital pathology workflow (or pixel pathway) can be understood as a combination of the errors related to each component. We propose a systematic assessment to decompose the total error across the individual components of the pixel pathway. Simply put, each workflow component could be assigned an individual importance factor for the overall performance, eventually leading to a mathematical or statistical model for risk assessment.

3) How to simplify regulation (FDA) and remove barriers for clinical implementation (CLIA) while ensuring safety and efficacy?

We need to identify a scientifically sound and efficient way to generate data demonstrating clinical utility and safety of digital pathology devices. For example, the real-world data (RWD) and real-world evidence (RWE) frameworks of the FDA[19] come to mind and would provide a logical starting point that also aligns with goals of the 21st Century Cures Act.
In addition, digital pathology and AI empowered devices are rapidly evolving technologies. These technologies will be subject to fast paced, iterative improvements and we are collectively tasked to create an environment that adapts to accelerating change via agile regulatory frameworks (see talk by George Poste). The FDA can solicit input from collaborative communities and/or a network of experts to supplement existing knowledge and gain rapid access to the relevant domain expertise[20]. Concretely, this also means delivery of scientifically sound and nimble regulatory tools (e.g., phantom slides, ground truth data sets, medical device development tools[21]) to enable assessment of appropriate performance (by the FDA or the laboratory).

Even if regulatory hurdles are removed, the field is reliant on data-driven approaches for implementation of digital pathology workflows to ensure alignment of intent and outcomes.