Study raises transparency concerns about clinical trials involving AI
The potential of artificial intelligence (AI) in clinical trials may have been overstated, according to researchers, who suggest drug studies involving the technology are being underreported.
The conclusion is based on research that identified discrepancies in how the results of studies including an AI algorithm in their clinical intervention have been disseminated in trial registries and journals.
For example, of the 28,248 peer-reviewed publications that reported the use of AI, only 1863 included a clinical trial registration ID. Likewise, only 101 of 1,106 completed AI-based studies logged on the ClinicalTrials.gov website had published results.
The findings raise concerns regarding the registration and reporting of AI-based clinical trial results, according to the authors.
“By examining trials that published results in both clinical trial registries and journals, we found limited evidence available across these dissemination platforms, accompanied by prevalent discrepancies, omissions, publication bias, and selective reporting. While registries and publications serve distinct roles, they also act as complementary sources for disseminating research findings.
“The study emphasizes the importance of utilizing multiple data sources, adhering to established reporting guidelines, and enhancing the accuracy and transparency of AI clinical trial evidence,” the authors wrote.
Protocols and publishing
They cite the SPIRIT-AI and CONSORT-AI initiative, which provide guidelines for protocols and reporting of trials involving AI developed by researchers, industry representatives, journal editors, lawyers, policy makers and regulators, as a publishing model.
“The growing field of AI in healthcare has led to the development of reporting standards to enhance research quality and transparency. For instance, the SPIRIT-AI and the CONSORT-AI emphasis on transparent reporting of clinical trials assessing the performance of AI in clinical settings.
“Both sets of guidelines stress the importance of clearly detailing AI interventions, comparisons with other treatments, and data processing methods to ensure scientifically rigorous and clinically meaningful results,” they wrote.
Exaggerated claims
Without such efforts, there is a danger that sponsors, CROs, and clinical researchers will overestimate the potential of AI and use it in place of human experts, according to the authors.
“Despite the enthusiasm for AI’s potential in healthcare, there is a shortage of solid clinical evidence underpinning AI-driven therapies. The exaggeration of AI performance becomes a major concern as it could potentially misguide clinical decisions with the overstated benefits.
“AI needs evidence-based validation, but currently, the quality of AI evidence is generally at a lower level, and there are concerns over ‘exaggerated’ claims of AI outperforming doctors,” the authors wrote.
DepositPhotos/tashatuvango