[Contribution] Challenges in legal and ethical aspects of AI and IoT-based medical devices

The contributor and Pharmaceutical Newspaper planned a series related to medical devices. In this contribution, we will examine the ethical and regulatory challenges of AI and IoT-based medical devices.

1. A new paradigm in medical devices

While the AI ​​craze represented by ChatGPT is spreading throughout society, artificial intelligence (AI) and Internet of Things (IoT) technologies are also being rapidly introduced in medical settings. It is no longer unfamiliar to see AI software that learns image data detect major diseases such as cancer and cardiovascular disease early, and monitor patients' ECG, blood sugar, and sleep patterns 24 hours a day through IoT wearable devices.

However, these innovative changes are accompanied by legal and ethical challenges. “Who is responsible if the diagnosis made by AI is wrong?”, “How safe is patient data collected through IoT devices?” Questions such as these have now become an unavoidable reality.

2. Characteristics of AI/IoT medical devices – products that are difficult to regulate under the existing regulatory system

AI algorithm’s learning ability:Once an existing medical device receives approval, its structure and performance are fixed. However, AI/IoT devices have fundamentally different characteristics. In other words, the performance of AI algorithms is updated as learning data increases, and sometimes the prediction method itself can change.

Frequent updates:Through the above learning, software patches, dataset replacement, and cloud updates are made, and AI medical devices change frequently accordingly. There is little change in approval requirements for existing medical devices after approval, and the existing Medical Device Act is enacted to suit these characteristics. However, AI medical devices may be difficult to regulate under existing approval systems due to frequent updates.

The two sides of IoT connectivity: Remote monitoring and cloud linkage are possible through continuous network connection, and the resulting convenience is considerable, but security risks also increase as a reaction.

Existence of areas that are difficult to regulate under the existing medical law-medical device law system: For existing medical devices, the doctor is the subject of the final diagnosis, so the doctor has the final responsibility. However, in the case of AI software that makes its own decisions based on learning data, medical devices can even perform diagnosis, making it unclear who takes ultimate legal responsibility.

These characteristics are factors that make it difficult to regulate AI/IoT medical devices, resulting in various legal and ethical issues.

3. Ethical challenges – issues of patient rights and responsibilities

go. Lack of transparency in the decision-making process

AI’s decision-making process is close to a ‘black box’. In fact, although research is still ongoing, it is said that it is difficult for humans to explain ChatGPT's thought process. This means that it is difficult for us to fully understand how AI thinks.

Patients and medical staff receive diagnoses from AI, but it is difficult to know how the results were derived. This may result in the patient's right to self-determination being violated or the location of legal responsibility becoming unclear.

me. Unclear responsibility

The following entities are involved in diagnostic activities using AI medical devices, but the distribution of responsibility among them is still unclear.

• Medical staff: Doctor who made the final diagnosis and treatment decision
• Manufacturer: Medical device company that designed the algorithm
• Hospital: Institution that introduced the device and used it on patients

all. Privacy and Security Risks

Real-time data transmission through IoT devicesThe risk of hacking inevitably follows along the process. If sensitive health information, such as a patient's electrocardiogram or blood sugar records, is leaked, the patient's moral rights may be seriously violated.

In addition, patients' health information is used to learn AI diagnostic software, but the contributor understands that there are still many cases of 'AI learning' without proper consent for the use of personal information. Although it has not yet become an issue, the issue of personal information infringement may also arise in this regard.

4. Regulatory challenges
In the case of Korea, software medical devices (SaMD) are regulated through the Medical Device Act and the recently enacted Digital Medical Products Act, and the regulatory sandbox system is used to simultaneously allow and verify new technologies temporarily. However, specific regulations on 'continuous learning' of AI algorithms, liability, and personal information protection issues mentioned above are lacking.

The US (FDA) discloses AI/ML SaMD-related guidelines on its website. The main feature of European Union (EU) regulation is the dual regulatory approach. The European Medicines Agency (EMA) manages AI-based medical devices under the Medical Device Regulation (MDR) and the Artificial Intelligence Act (AIA).

MDR specifies requirements for design, development, clinical evaluation, and post-marketing surveillance of medical devices, including AI-based medical devices, while AIA specifies risk-based classification, transparency, and human oversight requirements.

Medical AI companies must meet strict standards when entering the US and European markets to enter the market, so they need to prepare for this. In addition, domestic regulations are still being established, so it seems necessary to prepare in advance for regulations that will become stricter.

5. Conclusion

AI/IoT-based medical devices are already revolutionizing the lives of patients, and this trend is certain to accelerate further. However, the problem is that there is still a large gap between the speed of technological development and the speed of development of legal and ethical systems.

In particular, no one is able to clearly explain how to deal with the ethical and legal issues discussed above.

Medical device innovation is sustainable only when patient safety and rights are guaranteed, so ethical and regulatory discussions to ensure this are urgently needed. Therefore, the contributor believes that a forum for discussion in which regulatory agencies, industry, medical professionals, and the legal community participate together is needed.

Meanwhile, it is clear that government regulations related to AI/IoT-based medical devices will be strengthened as a general trend. Some regulations may require considerable technological difficulty, and it is believed that related industries need to prepare proactively for this.

Contributor: Attorney Ilhyung Lee (ilhyunglee@naver.com)
Passed the lawyer/pharmacist/patent attorney/American public accountant (Maine) exam
(Former) Attorney at Celltrion Domestic Legal Team
(Current) Head of Daeryun Medical Pharmaceutical Group/Pharmaceutical Bio Healthcare Center, Law Firm (Limited)

[View full article]
[Contribution] Legal and ethical challenges of AI and IoT-based medical devices