Your algorithm predicts cancer more accurately than a doctor. Congratulations. Now let’s see whether you can legally sell it in Europe, because the AI Act, MDR and GDPR have plenty to say about that.
Polish and international startups are developing AI systems that analyse histopathology images, conduct autonomous cognitive behavioural therapy sessions, predict the risk of a heart attack or diagnose Alzheimer’s from a 10-second voice recording. The technology works. The problem is that before such a product can reach patients in the EU, it must navigate three overlapping regulatory regimes: the AI Act (EU 2024/1689), the MDR (EU 2017/745) and the GDPR.
If you are building an AI product for the healthcare sector, diagnostics, clinical decision support, a therapeutic chatbot, or a bionic prosthesis controlled by muscle signals, this article will explain what you need to do before you start selling it. I will outline the compliance timeline, classification criteria and specific steps worth taking right now.
Key terms you need to know
Before we get into the details, let’s clarify the terminology. The three regulations use their own definitions, and understanding them is essential to classifying your product correctly.
| Term | What it means | Where it is defined |
|---|---|---|
| High-risk AI system | An AI system that, due to its intended purpose, may pose a significant risk to the health, safety or fundamental rights of natural persons | Art. 6 AI Act |
| MDSW (medical device software) | Software that independently meets the definition of a medical device, it has a diagnostic, therapeutic or monitoring purpose in relation to an individual patient | Art. 2(1) MDR 2017/745 |
| Provider of an AI system | An entity that places an AI system on the market or puts it into service under its own name, even if it commissioned a subcontractor to develop it | Art. 3 AI Act |
| Deployer | An organisation (e.g. a hospital or clinic) that uses an AI system as part of its professional activities | Art. 3 AI Act |
| Special categories of data | Health, biometric, genetic and other data listed in Art. 9(1) GDPR, their processing is generally prohibited, subject to the exceptions in Art. 9(2) | Art. 9 GDPR |
| Conformity assessment | A procedure for verifying whether a product meets regulatory requirements, depending on the risk class, it may require the involvement of a notified body (an independent auditor) | Art. 43 AI Act, Annex IX MDR |
One caveat: the European Commission is expected to issue guidelines with practical examples of high-risk AI systems no later than 2 February 2026. Until then, companies must assess the classification of their products themselves, and document that assessment.
Why your AI product is probably a high-risk system
The AI Act classifies AI systems according to their level of risk. For the healthcare sector, this means that the vast majority of AI products fall into the high-risk category, through two independent routes.
Route 1: through the MDR (Art. 6(1) AI Act)
An AI system is automatically classified as high-risk if it meets both of the following conditions:
- It is a safety component of a medical device covered by MDR 2017/745 (or IVDR 2017/746) or is itself such a device.
- The device in question is subject to a third-party conformity assessment by a notified body, which applies to all Class IIa and higher medical devices.
Rule 11 of Annex VIII MDR classifies medical software as follows:
| Intended purpose of the software | MDR class | High-risk system under the AI Act? |
|---|---|---|
| Provides information used to make diagnostic or therapeutic decisions | IIa | Yes |
| An incorrect decision may cause a serious deterioration in health or require surgical intervention | IIb | Yes |
| An incorrect decision may result in death or an irreversible deterioration in health | III | Yes |
| Monitoring vital physiological parameters where a change could endanger the patient | IIb | Yes |
| Other medical software | I | No (no mandatory assessment by a notified body) |
Conclusion: if your AI software supports patient diagnosis, therapy or monitoring and meets the definition of MDSW, it is very likely to fall into at least Class IIa. And that automatically makes it a high-risk system within the meaning of the AI Act.
Route 2: through Annex III AI Act (Art. 6(2))
Even if your AI system is not a medical device, it may be classified as high-risk under point 5 of Annex III. This applies to AI systems used for:
- Assessing individuals’ eligibility for public and private healthcare services
- Risk assessment and pricing in life and health insurance
- Classifying and prioritising emergency calls and dispatching emergency medical response teams
- Patient triage in emergency healthcare
When the classification exemption may apply to you
Art. 6(3) AI Act provides that a system listed in Annex III is not high-risk if it does not pose a significant risk of harm and meets one of the following conditions:
- It performs a narrow procedural task
- It improves the result of a previously completed human activity
- It detects patterns without replacing human assessment
- It performs preparatory tasks
But note: the exemption does not apply when the system profiles natural persons. If you conclude that your system is not high-risk, you must document that assessment before placing it on the market and register the system in the EU database.
For example, an AI system used to analyse the quality of diagnostics internally at a hospital, without affecting current clinical decisions, could potentially qualify for this exemption. A therapeutic chatbot conducting autonomous CBT sessions probably would not.
Three regulations, one product, how the AI Act, MDR and GDPR overlap
If your healthcare AI product qualifies as a medical device, you are simultaneously subject to three regulatory regimes. You cannot choose one. You must comply with the requirements of all three.
How the obligations compare
| Area | AI Act | MDR | GDPR |
|---|---|---|---|
| Risk management | Art. 9, an iterative system throughout the entire lifecycle | Annex I, device risk analysis | Art. 35, DPIA for large-scale processing |
| Technical documentation | Art. 11, Annex IV, model description, training data, metrics | Annexes II and III, device technical documentation | Art. 30, record of processing activities |
| Data quality | Art. 10, relevant, representative and error-free training data | Art. 61, clinical evaluation, clinical data | Art. 5, data minimisation and purpose limitation principles |
| Human oversight | Art. 14, the doctor must be able to disregard, override or switch off the system | Rule 11, the diagnostic/therapeutic decision remains with the doctor | Art. 22, the right not to be subject to automated decision-making |
| Post-deployment monitoring | Art. 72, continuous collection and analysis of performance data | Arts. 83-86, post-market surveillance and vigilance system | Art. 5(1)(e), storage limitation |
| Conformity assessment | Art. 43, internal control or notified body | Annexes IX-XI, procedures involving a notified body | Not directly applicable (but a DPIA is mandatory) |
The good news: Recital 46 of the AI Act expressly provides for the possibility of integrating MDR and AI Act conformity assessment processes to avoid duplicating the burden. A provider may combine technical documentation, testing procedures and audits. This does not eliminate the obligations, but it can help optimise costs and time.
Health data in ML model training, what the GDPR says
By definition, medical AI systems process special categories of data: health, biometric and genetic data. Art. 9(1) GDPR prohibits such processing, unless one of the grounds under Art. 9(2) applies.
The legal grounds most commonly used by healthtech startups:
| Ground under Art. 9(2) GDPR | When it applies | Limitations |
|---|---|---|
| Point (a), explicit consent | The patient knowingly consents to the processing of data for a specific purpose | Problematic in employment relationships (imbalance between the parties); must be specific and freely given |
| Point (h), diagnostic and healthcare purposes | Processing necessary for diagnosis, prevention or treatment | Requires processing to be carried out by or under the responsibility of a person subject to professional secrecy |
| Point (i), public interest in the area of public health | Detection, diagnosis and control of diseases | Requires a basis in EU or Member State law |
| Point (j), scientific research or statistical purposes | Training an AI model on historical data for research purposes | Requires appropriate safeguards (pseudonymisation, encryption, access restrictions); Art. 89 GDPR |
Note Art. 469b of the Polish Law on Higher Education and Science: entities within the higher education system may process special categories of data for scientific research and development work, provided that the data is pseudonymised and access is restricted. But this applies to scientific institutions, not every startup.
In its December 2024 opinion, the European Data Protection Board (EDPB) stated that an AI model may be considered anonymous only when both direct and indirect identification of individuals from the training data is highly unlikely. For systems trained on medical data, diagnostic images, voice recordings and biometric signals, achieving full anonymisation is difficult.
Therefore, if you train a model on patient data, conduct a data protection impact assessment (DPIA) for every process involving health data. Identify the legal ground under Art. 9(2) GDPR. Document the technical and organisational safeguards.
Contact us, we will help you assess which legal ground is appropriate for your data processing model.
Timeline: how long you have to comply
Not all obligations take effect at the same time. The timeline below shows what already applies and what is still ahead.
| Date | What takes effect | Who it applies to |
|---|---|---|
| 2 February 2025 | Prohibitions on unacceptable practices (Art. 5 AI Act); AI literacy obligation (Art. 4) | All providers and deployers of AI systems |
| 2 August 2025 | Rules on market surveillance authorities, general-purpose AI (GPAI) models and penalties | Providers of GPAI models; Member States (designation of authorities) |
| 2 February 2026 | European Commission guidelines with practical classification examples | Everyone (guidance document) |
| 2 August 2026 | Full application of the rules for high-risk AI systems | Providers and deployers of medical AI systems |
| 9 December 2026 | Deadline for implementing the new Product Liability Directive (PLD 2024/2853) | Manufacturers of products, including software and AI systems |
The draft Polish Act on Artificial Intelligence Systems provides for the establishment of the Commission for the Development and Safety of Artificial Intelligence (KRiBSI) as the national market surveillance authority. KRiBSI will have the power to impose administrative fines of up to EUR 35 million or 7% of worldwide annual turnover. The commission will include representatives of UOKiK, KNF, KRRiT, UKE, the National Labour Inspectorate and the medical devices regulator.
As a rule, proceedings before KRiBSI will be conducted in a single instance, with appeals heard by the Court of Competition and Consumer Protection.
Who is liable when AI makes an incorrect diagnosis
This is one of the most common questions asked by healthtech founders, and the answer is not straightforward.
Current legal framework
The proposed AI Liability Directive (AILD) was withdrawn from the European Commission’s legislative agenda in February 2025. This means that there is not yet a dedicated EU-level liability regime for damage caused by AI systems.
Instead, the following rules apply:
- The new Product Liability Directive (PLD 2024/2853), extends the definition of a product to include software and AI systems. Implementation deadline: 9 December 2026. The injured party does not need to prove fault on the part of the manufacturer, it is sufficient to demonstrate that the product was defective and that the defect caused the damage.
- General provisions of Polish law, Art. 415 of the Civil Code (fault-based liability), Art. 430 of the Civil Code (an employer’s liability for a subordinate), Art. 435 of the Civil Code (strict liability for an enterprise powered by forces of nature).
- Liability for a dangerous product, Arts. 449¹-449¹¹ of the Civil Code.
Who is liable in each scenario
| Scenario | Who is liable | Legal basis |
|---|---|---|
| A defective algorithm leads to an incorrect diagnosis | The AI system provider (manufacturer) | PLD 2024/2853 (from XII 2026); Art. 449¹ of the Civil Code |
| A doctor ignores a correct AI recommendation or fails to verify an incorrect one | The doctor / healthcare facility | Arts. 415, 430 of the Civil Code; professional liability |
| A hospital modifies the AI system (further trains the model, changes its intended purpose) | The hospital becomes a provider within the meaning of the AI Act | Art. 3 AI Act, change of role |
| Lack of human oversight over the AI system | The deployer (healthcare facility) | Art. 26 AI Act, obligations of the deployer |
Worth knowing: a breach of AI Act requirements (e.g. no technical documentation or no risk management system) may serve as an additional argument in a civil liability dispute, as evidence that due care was not exercised.
What to do before your product reaches the market, a list of priorities
Below is a specific sequence of actions for the founder or CTO of a healthtech/medtech startup. Each step corresponds to a regulatory requirement.
Actions you must take (must do)
- Classify the AI system under Annex III AI Act. Determine whether your product is a high-risk system. If it is a Class IIa or higher medical device, the answer is yes. Document this assessment.
- Verify whether your software meets the definition of a medical device (MDSW) under the MDR. Check whether the software has a diagnostic, therapeutic or monitoring purpose in relation to an individual patient. If it does, initiate the CE certification process with the involvement of a notified body.
- Conduct a DPIA for every process involving patients’ health data. Identify the legal ground under Art. 9(2) GDPR. Document the safeguards: pseudonymisation, encryption and access restrictions.
- Implement a human oversight mechanism. The doctor must be able to verify the system’s recommendation, reject it or switch off the system. This is a requirement under both the AI Act (Art. 14) and the MDR.
- Ensure AI literacy within the team. Art. 4 AI Act has applied since 2 February 2025, you must ensure an appropriate level of AI expertise among people who interact with the system or are responsible for its operation.
Actions you should take (should do)
- Prepare technical documentation for the AI system in accordance with Annex IV AI Act. Include a description of the model, training data, performance metrics, risk analysis and mitigation measures. Consider integrating it with the MDR documentation, Recital 46 of the AI Act permits this.
- Implement an event logging system. Arts. 12 and 19 AI Act: automatic logging throughout the entire lifecycle, with records retained for at least 6 months.
- Train the product and development teams. Explain the roles: AI system provider, deployer and distributor. Each role carries different obligations. In particular, if a customer (hospital) modifies your system, it may become a provider and assume the full range of provider obligations.
- Plan post-market monitoring (Art. 72 AI Act). Actively and systematically collect data on system performance, analyse incidents and take corrective action.
- Check whether you can use a regulatory sandbox. Art. 57 AI Act provides for the creation of regulatory sandboxes, controlled environments for testing innovative AI systems under the supervision of an authority. Member States must establish at least one sandbox by 2 August 2026. The Polish draft legislation provides that KRiBSI will have the power to establish them.
How much does it cost, a startup perspective
There is no single figure. But the available data allows us to estimate the scale of the burden.
| Cost category | Estimated range | Source |
|---|---|---|
| MDR certification for a Class III device | EUR 1-4 million | Industry data, MedTech Europe reports |
| MDR compliance as a percentage of revenue (medium-sized company) | 8-15% | Industry data |
| AI Act conformity assessment (high-risk systems) | Tens to hundreds of thousands of euros | Expert estimates (no reliable market data) |
| DPIA for large-scale health data processing | Internal cost or external audit | Depends on scale |
VC market survey data:
- 73% of venture capital funds expect the AI Act to reduce the competitiveness of European AI startups
- 50% of startups believe that the AI Act will slow down innovation
- 16% of startups are considering moving their operations outside the EU
The time required to implement compliance is estimated at 12-24 months, which, given the deadline of 2 August 2026, means that the window for starting work closes in mid-2025.
We can help you
Building an AI product for healthcare requires three regulatory paths to be planned in parallel. Each has its own deadlines, documentation and procedures. Overlooking any one of them may prevent the product from being placed on the EU market or expose the company to fines of up to EUR 35 million.
We help healthtech and medtech startups structure these areas:
- AI system classification, assessing whether your product is a high-risk system under the AI Act and/or a medical device under the MDR
- Compliance roadmap, an action plan with a timeline, budget and allocation of responsibilities
- Technical documentation, preparing documentation compliant with Annex IV AI Act and integrated with MDR documentation
- DPIA and data strategy, identifying legal grounds for processing health data and implementing technical and organisational safeguards
- Agreements with subcontractors and partners, defining the roles of provider, deployer and distributor within the value chain
- Regulatory sandbox support, assessing eligibility for an AI Act sandbox and preparing application documentation
We combine IT/AI/IP law with data protection and medical device law, because your product is subject to these regulations collectively, not separately.
Before your algorithm reaches patients, make sure it is legally ready
The AI Act, MDR and GDPR create three overlapping regulatory layers for medical AI systems. The AI Act provisions for high-risk systems will apply in full from 2 August 2026, but some obligations, the prohibitions on unacceptable practices and AI literacy, have already applied since February 2025.
If you are building an AI product for the healthcare sector, your window for preparation is closing quickly. System classification, a DPIA, technical documentation, a human oversight mechanism and a training data strategy are not tasks to leave until the final month before the deadline.
Check whether your AI product is ready for the AI Act. Contact us, we will conduct a system classification audit and prepare a compliance roadmap tailored to your company’s stage of development.
Frequently asked questions
Is my healthcare AI product a high-risk system under the AI Act?
If your AI software supports patient diagnosis, therapy or monitoring and meets the definition of a Class IIa or higher medical device under the MDR, then yes, it is automatically a high-risk system under Art. 6(1) AI Act. Even if it is not a medical device, it may be classified as high-risk under Annex III (e.g. triage systems or systems that assess eligibility for healthcare services). The exemption under Art. 6(3) applies to narrowly defined procedural tasks, but not when natural persons are profiled.
Does AI software that supports diagnosis require CE certification as a medical device?
Yes, if it meets the definition of MDSW under Art. 2(1) MDR, meaning that it has a diagnostic, therapeutic or monitoring purpose in relation to an individual patient and performs operations on data that go beyond simple storage or communication. Rule 11 of Annex VIII MDR classifies such software as at least Class IIa, which requires the involvement of a notified body in the conformity assessment. Without CE certification, you cannot legally place the product on the EU market.
What data can I collect from patients to train an AI model, and on what legal basis?
Health and biometric data are special categories of data under Art. 9 GDPR. You need both a legal basis under Art. 6(1) GDPR and an exception to the prohibition under Art. 9(2), most commonly: explicit consent (point (a)), diagnostic and healthcare purposes (point (h)) or scientific research (point (j), with the safeguards required by Art. 89 GDPR). Conduct a DPIA for every process involving health data. In its December 2024 opinion, the EDPB stressed that an AI model may be considered anonymous only when identifying individuals from the training data is highly unlikely.
How long do I have to comply with the AI Act if my product is already on the market?
The rules for high-risk AI systems will apply in full from 2 August 2026. However, the prohibitions on unacceptable practices and the AI literacy obligation have applied since 2 February 2025. Implementing compliance is estimated to take 12-24 months, which means that adaptation work should begin as soon as possible.
Who is liable when AI makes an incorrect diagnosis, the system provider, the hospital or the doctor?
It depends on the scenario. The AI system provider is liable for product defects (under PLD 2024/2853 from December 2026 and the Polish provisions on liability for dangerous products). The doctor and healthcare facility are liable for failing to verify the system’s recommendation or for ignoring a correct recommendation. A hospital that modifies an AI system (e.g. further trains the model on its own data) becomes a provider within the meaning of the AI Act and assumes the full range of provider obligations. A breach of AI Act requirements may serve as an additional argument in a civil liability dispute.
Can I test an AI system on patients as part of a pilot programme without full certification?
Art. 57 AI Act provides for regulatory sandboxes, controlled environments for testing innovative AI systems under the supervision of an authority. Member States must establish at least one sandbox by 2 August 2026. Art. 54 AI Act regulates the processing of personal data in a sandbox, while Art. 58 covers testing outside a sandbox. Under the MDR, clinical investigations of medical devices are subject to separate procedures (Art. 61 et seq. MDR). Conducting a pilot without any regulatory basis creates exposure to product liability and administrative fines.
