A design framework for operationalizing trustworthy artificial intelligence in healthcare: Requirements, tradeoffs and challenges for its clinical adoption

The healthcare sector stands as one of the most promising fields where Artificial Intelligence (AI) can deliver groundbreaking advances in the diagnosis and prognosis of diseases and patient care. Over the past decade, the increase of electronically recorded medical data and advancement in computing capability have paved the way for new data-driven solutions that by using, amongst others, medical images, omics, biosignals, and electronic health record data are close to the capabilities of medical experts in detecting and predicting diseases. Despite their potential and revolutionary impact, the deployment of AI solutions in the healthcare sector faces significant barriers, more related to other issues, such as human-technology acceptance, ethics, and regulatory considerations, than technological constraints. To facilitate clinical adoption, these data-driven solutions must adhere to the Trustworthy AI (TAI) paradigm, which addresses critical aspects such as human agency and oversight, algorithmic robustness, privacy and data governance, transparency, bias and discrimination avoidance, and accountability. However, the inherent complexities of the healthcare sector, as reflected in its diverse processes (including screening, diagnosis, prognosis, and treatment) and the variety of stakeholders involved (clinicians, patients, healthcare providers, and regulators), make the adoption of TAI requirements challenging. This work contributes to the field of TAI in healthcare with a design framework that allows developers of medical AI systems to focus on the various requirements and characteristics that these solutions must meet for the healthcare stakeholders to consider these systems trustworthy. Thus, for each stakeholder identified across various healthcare processes, we propose a disease-agnostic collection of requirements that medical AI systems should incorporate to adhere to the principles of TAI. To make the framework operational, we specify which aspects should be measured and evaluated to ensure these principles are adequately addressed. Furthermore, we also discuss the challenges and tradeoffs that could emerge by upholding these requirements in the healthcare domain. Since cardiovascular diseases represent one of the most active and high-impact areas in the development of data-driven medical models, we use this clinical domain to illustrate how the different TAI principles have been addressed in practice and to highlight the specific challenges encountered when applying them to medical AI systems in cardiovascular healthcare.