Many medical AI developers unaware of key AI regulations, study finds

Researchers surveyed 122 developers working on medical AI in Singapore, China, Hong Kong and the UK. They described the work as one of the first surveys of its kind to examine regulatory awareness across several regions. The results were published in the journal npj Digital Medicine.

Of those questioned, 57% said they knew of at least one regulatory framework. Awareness varied by seniority and workplace: senior developers and those employed outside universities reported knowing more frameworks than junior colleagues and academic researchers.

Two-thirds of respondents worked for organizations that had adopted no AI governance framework at all, according to the study. Developers whose employers had adopted one reported greater familiarity with the relevant regulations than those whose employers had not.

At the same time, most respondents said they should answer for the systems they create. They saw that responsibility as shared, with guidance from government or institutional ethics bodies and cooperation with users and the organizations that supplied the data used in development.

Wilson Goh, assistant professor at NTU's Lee Kong Chian School of Medicine and chief data scientist at the university's Centre of AI in Medicine, who co-led the study, said developers are well placed to judge data quality, the limits of their models and risks including bias and hallucinations. He described the sense of responsibility among developers as reassuring but called the awareness gaps a concern, adding that a more active approach is needed so AI models do not put patients or public confidence at risk.

Among the frameworks cited in the study are the European Union's AI Act and Singapore's Artificial Intelligence in Healthcare Guidelines. The researchers said limited familiarity with such safeguards could have consequences for patients and public trust once the tools are introduced into clinical practice, a point they said is particularly relevant in countries investing heavily in digital technology.

To close the gap, the researchers proposed that universities and other training institutions teach regulatory frameworks as part of developer education. Inside companies, they suggested structured mentoring between experienced and junior staff to help newcomers better understand compliance requirements.

They also called on national regulators to take a more active role in promoting compliance and to work toward aligning requirements across jurisdictions, drawing on the experience of developers themselves.

Joseph Sung, NTU's senior vice-president for health and life sciences, director of the Centre of AI in Medicine and dean of the medical school, who led the study, said the challenges of introducing AI into healthcare cannot be resolved by one group alone. Collective ownership of the consequences of deployment is needed to address patient safety, data privacy and the long-term reliability of the technology in medicine, he said.

AI is being applied in a growing number of clinical tasks, from analyzing medical images to estimating a patient's risk of disease. As healthcare systems increasingly adopt these tools, the researchers argue that improving developers' understanding of regulatory requirements will be as important as advancing the technology itself.

The survey follows earlier work by the same team, which reported that gastroenterologists broadly trust and accept the use of AI tools in clinics and hospitals.

Earlier, Qazinform reported that AI could contribute to large-scale cyberattacks, weapons development, rising unemployment, environmental damage and the concentration of power among a small number of companies.