Why Medical AI Deployment Requires a Risk File
Holding
A medical AI system should not be deployed on the strength of an accuracy table alone. It requires a risk file that connects intended use, hazards, controls, verification, validation, monitoring, and corrective action.
Authority
ISO 14971 describes risk management as a lifecycle process for medical devices. FDA quality system requirements and EU medical device rules similarly place safety, performance, and documentation duties on the manufacturer. The point is not paperwork. The point is traceable control over foreseeable harm.
Issue
AI failure can occur without a crash. A model can be overconfident for a subgroup, sensitive to missing fields, unstable under a new scanner, or inappropriate for a population outside its validation set. If those hazards were never identified, the team cannot show that controls were selected deliberately.
Resolution
The risk file should define the intended medical use, the user, the patient population, the data assumptions, the hazards, the risk controls, and the evidence that each control works. For AI systems, the file should also include drift criteria, human oversight points, bias assessment, fallback mode, and post-deployment surveillance.
Evidence Package
The deployable package includes a hazard analysis, risk-control matrix, validation evidence, residual risk review, monitoring plan, incident criteria, and change-control pathway. The practical rule is simple: if a risk cannot be traced to a control and evidence, it has not been governed.