- AI in a medical device is a clinical claim, not a model feature. You validate the intended purpose, not the demo.
- The MDR class decides whether the AI Act's high-risk rules apply. If the device needs a notified body, they apply from 2 August 2028.
- Data, bias and model changes are regulatory questions. Plan them in the quality system from the start.
AI in medical devices is allowed and in use, but a model that works in a notebook is not a product you can use on patients. When AI software has a medical purpose, it must meet the MDR or IVDR, be developed under IEC 62304, be risk-managed under ISO 14971 and be validated for its intended purpose. If the device needs a notified body, the EU AI Act’s high-risk requirements also apply from 2 August 2028.
This article covers the work between the promising model and the CE-marked version: classification, rules, validation, data, bias and updates.
When is AI software a medical device?
AI software is a medical device when the manufacturer intends it to diagnose, prevent, monitor, predict or treat disease in individual patients. The intended purpose, not the technology, determines whether the software is a medical device. An AI model that plans staff rotas is not a medical device. An AI model that flags suspicious findings on an X-ray is.
The class follows from Rule 11 in Annex VIII of the Medical Device Regulation (EU) 2017/745. Software that provides information for diagnostic or therapeutic decisions is Class IIa by default, and Class IIb or III if a wrong decision can cause serious deterioration or death. Software that monitors physiological processes is Class IIa. All other software is Class I. In vitro diagnostic software is classified under the IVDR’s own rules.
The European guidance MDCG 2019-11 rev.1, revised in June 2025, works through the rules with examples, including AI. National authorities such as the Danish Medicines Agency publish their own guidance for manufacturers of software and apps. Our roadmap from idea to CE marking shows how the class shapes time and budget.
Which rules apply to AI in medical devices?
AI in a medical device must comply with the same rules as other medical software, plus the EU AI Act when the device is high-risk. Most of the work overlaps: a good requirements process, risk assessment and test strategy can support compliance with all of the frameworks if they are properly documented from the start.
| Framework | What it covers |
|---|---|
| MDR / IVDR | Whether the product is a medical device, its class, clinical evidence and post-market surveillance |
| IEC 62304 | The software life cycle: requirements, architecture, verification, maintenance and SOUP |
| ISO 13485 | The quality management system, including design controls and change control |
| ISO 14971 | Risk management, including risks from model errors and bias |
| EU AI Act | Data governance, documentation, transparency, human oversight and robustness for high-risk AI |
| GMLP | Ten principles of good machine learning practice, published by the IMDRF in N88 (January 2025) |
When does the EU AI Act apply to medical devices?
The EU AI Act (Regulation (EU) 2024/1689) makes AI in a medical device high-risk when two conditions are met: the AI system is the device itself or a safety component of it, and the device needs a notified body assessment under the MDR or IVDR. Software in Class IIa and above is therefore usually in scope. Self-declared Class I software is not.
The implementation timetable has changed:
- 2 February 2025: Prohibited AI practices and the AI literacy rules apply. Since the 2026 amendment, providers and deployers must take measures that support AI literacy among their staff.
- 2 August 2026: The transparency rules in Article 50 apply, including the requirement that people must know they are interacting with AI.
- 2 August 2028: The high-risk requirements apply to AI in products such as medical devices. The date was originally 2 August 2027, but was postponed by the Digital Omnibus Regulation (EU) 2026/1744, published on 24 July 2026 and in force from 27 July 2026.
The joint MDCG and AI Board guidance MDCG 2025-6 from June 2025 explains how the rules fit together. In short, the AI Act does not change the MDR class, the notified body assesses both sets of rules in one procedure, and the AI requirements can be built into the existing quality system and technical documentation.
How do you validate an AI model in medical software?
To validate an AI model, gather evidence that the product does what you claim for its intended users and patients. AI adds a layer: the model’s behaviour depends on data, and data changes.
A practical order:
- Define the intended purpose. Who uses the product, for which decision, and for which patient population? That is what you validate.
- Lock the data and the model. Version the training data, code, hyperparameters and the final model, so the result can be reproduced.
- Test on independent data. Test data must not overlap with training data and should reflect clinical reality, ideally collected at sites other than those used to collect the training data.
- Document performance and errors. Report results overall and for relevant subgroups, using clinical measures such as sensitivity and specificity rather than a single accuracy score.
- Test the workflow. Usability under IEC 62366-1: does the clinician understand the output, and what happens when the model is uncertain?
- Monitor after release. Data drift and falling performance must be caught before they become a clinical problem, as part of post-market surveillance.
What does the data need to look like?
Training and test data must be of sufficient quality, representative of the intended patient population and documented. If the data cannot be documented, the model cannot be validated.
- Quality. Missing values, different units and signal noise must be handled explicitly and described.
- Representativeness. The data must cover the population the product will be used on, not only the patients who were easy to recruit.
- Traceability. Every training and test set must be traceable to a version, a source and a processing step.
- Data protection. Health data is a special category of personal data under GDPR Article 9. You need a legal basis, pseudonymisation and access control, and a data processing agreement with the vendors who process the data.
How do you handle bias in medical AI?
Handle bias in medical AI through risk management: it is a safety issue, not only an ethical one. If the model performs worse for a patient group, that is a difference in safety and performance that must be handled in risk management.
Treat bias as part of the design:
- Map which groups are under-represented in the data, for example by age, sex, ethnicity or device type.
- Test performance separately for the relevant groups, with acceptance criteria set in advance.
- Document the decisions made to address bias and the limitations included in the instructions for use.
- Monitor whether the distribution changes after release.
No technical fix removes all bias. You can make it visible, manage it and document it.
What happens when the model needs an update?
A retrained AI model is a change to the device. Under the MDR you must assess whether the change is significant, and a significant change to Class IIa software or higher must be approved by the notified body before release.
The AI Act allows for planned change: changes that are described in advance and included in the technical documentation at the initial conformity assessment are not a substantial modification. In the US, the FDA has had final guidance on Predetermined Change Control Plans (PCCP) for AI-enabled devices since December 2024. According to the FDA’s current PCCP guidance (August 2025), a PCCP describes the planned modifications, how they will be developed and validated, and their impact, and the FDA reviews it as part of the marketing submission.
In practice, most AI products are released with a locked model. If the model is meant to keep learning, the change protocol, the tests and the limits on what may change must be in place from the start.
What we see in practice at bon.do
In our work building software around AI in regulated settings, traceability and human oversight are recurring concerns. We built the backend for a CE-marked medical device in which a seismographic AI algorithm estimates VO₂max from the vibrations of the heart. The requirement was to make every result traceable to the raw recording, the device, the operator and the subject. That traceability is what makes the algorithm’s output usable in an audit.
On QualiBooking, a clinic platform that is not a medical device, AI notes run through a provider layer, and the practitioner reviews and signs every note. Same principle: the model can be swapped, and a human stays in control.
Whether you build in-house or with a partner, you should be able to answer:
- Who owns the data, the model and the code?
- Can the model be replaced without rebuilding the rest of the product?
- How are model changes documented and approved?
- Who monitors the model in use, and what triggers action?
These questions will arise during due diligence or an audit.
Next step
Start with a narrow clinical problem and confirm the class before you build. Build the smallest version that can be tested, document the data, the model and the decisions, and choose an architecture where the model can be swapped without rebuilding the whole product. Read more about ISO 13485 and the quality system and GDPR and health data.
If you want a second pair of eyes on your AI product, book a free project assessment. We go through the architecture, data and documentation with you, with no obligation. You can also read how we build regulated health and life science software.
Key facts
Questions we get asked
