Will Artificial Intelligence revolutionise clinical trials? - INDUSTRY VOICES

Artificial Intelligence and machine learning present a vast array of opportunities and challenges for the future of clinical trials, yet a number of questions remain. How can AI be used to improve the clinical trial design process? What are the main challenges of implementing AI and what advancements have been made? How can we think about communication and collaboration between technology and clinical experts? We asked three industry professionals for their insights into how AI will impact clinical trials and how they are currently working with it.

How can AI be used to improve the clinical trial design process?

"One in five trials fail because they cannot recruit enough patients. Others are delayed due to recruitment challenges and many require protocol modifications as they learn that the target population simply does not exist in sufficient numbers at their sites. Powerful applications of AI for trial design include defining cohorts that exist and identifying where they may be for site selection."

- Jennifer Goldsack, Executive Director at Digital Medicine Society (DiMe)
– follow @goldsackjen and @_DiMeSociety

"At Janssen, AI, also known as ML (Machine Learning), and Predictive Analytics is used in many aspects of clinical trial design, such as virtual control arms, site selection/feasibility assessment and AI-assisted monitoring and audits.  Our group focuses on the Audit / Finding / Corrective-And-Preventive-Action / Root Cause data which is largely free text. We are developing Natural Language Processing (NLP) and Machine Learning / Deep Learning approaches to search, cluster, classify and find anomalies in this free text. The application of AI in R&D Quality & Compliance is an integral part of applying Quality by Design (QbD) principles in our clinical studies."

- Deepak Bandyopadhyay, Associate Director of Quality Analytics at Janssen

"AI has great potential to improve the entire clinical trial process. When looking at the trial design process AI technologies can collect increasing amounts of data from current and past trials and extract meaningful patterns of information to help inform and improve the new trial design.

When designing a trial an area of focus is often patient recruitment and engagement. AI can be used to support the engagement of a clinical trial participant. For example, AI can help tailor the patient's journey, ensuring they receive optimal messages, reminders, and alerts depending on their demographics, behaviours, and usage patterns. Used well, AI can help reduce dropout rates while ensuring the patient has a great experience during the trial."

- Bruce Hellman, CEO and Co-Founder of uMotif - follow @umotif

What are the main challenges of AI and what advancements have been made?

"Algorithms are only as good as the underlying data. Clinical data may be incorrect, for example, due to inaccurate data entry, poorly-planned EHR structure, or incomplete data. Even when data is complete and correct, it may be biased with respect to representation, measurement, evaluation, and aggregation. Algorithms themselves may also be biased, and recognizing these biases is not always easy. Even if race and gender attributes are excluded from datasets, there is no guarantee that there won’t be racial and gender biases in their analyses. This is an example of machine learning exploiting variables that are not directly observed, only inferred. (These are what’s called “latent variables.”)"

- Jennifer Goldsack, Executive Director at Digital Medicine Society (DiMe)
– follow @goldsackjen and @_DiMeSociety

"There are several major challenges from my experience at Janssen:

• Free text data are not amenable to traditional Machine Learning – bridge them with Natural Language Processing.
• Reconcile the quality culture and expectations (where we demand 99.9% accuracy) with the ML reality (a 99.9% accurate model is likely overfit to the training set – models that generalize successfully have lower accuracy say over 80%)
• Data preparation, collation and labelling prior to modelling makes the difference between good and bad models
• Shift the culture from relying on looking at aggregate statistics only to drilling down visually into your data and predictions for deeper understanding – as an example look up “Datasaurus” on Google.
• Change management challenges – established ways-of-working laid down in SOPs and used constantly by staff are hard to replace with an AI tool, even when the existing way is suboptimal and the AI tool offers clear benefits.  So don’t underestimate the time that needs to be spent on educating, supporting, coaching and convincing users.
• Operational challenges – e.g. need a GPU cluster for deep learning, challenges in deploying, porting and reproducing models.
• Validation challenges – shift the culture to match validation rigor with model risk, when using results of predictive models for Quality & Compliance activities and regulatory filings.

However, there are also some exciting new advancements:

• From Word Embeddings to Language models to Graph-based concept models, NLP and its advanced cousins NLU (Understanding) and NLG (Generation) are coming of age and maturing.
• AI and data science overall have really springboarded on the Open Source movement to overtake the limited capabilities in proprietary vendor software (“closed source”) – most of the new advanced technologies are available as libraries within popular languages (R/Python) for anyone with programming skill to incorporate into custom solutions. This has catalysed the growth of the field."

- Deepak Bandyopadhyay, Associate Director of Quality Analytics at Janssen

"AI has significantly advanced in recent years with exciting applications - for example in enhancing radiology services. The digitisation within life sciences organisations is revolutionising processes across the entire industry. The huge amount of data collected during clinical trials can now be analysed and patterns emerge that only AI can detect.

As with many technologies, a challenge for AI is to ensure that it solves a real problem, or enhances a solution in a meaningful way, not only deploying technology for its own sake. AI needs to enhance how clinical trials are conducted and better support the people-based solutions that are such an integral part of the industry.

Another challenge is AI needs to earn the understanding and trust of the life sciences professionals. When dealing with data that has the potential of affecting people’s lives and wellbeing the AI technology has to be robust and secure."

- Bruce Hellman, CEO and Co-Founder of uMotif - follow @umotif

How can we think about communication and collaboration between technology and clinical experts? How does it/should it work?

"Our organization, the Digital Medicine Society (DiMe), exists to bring individual experts at the intersection of technology and clinical together. To improve communication and collaboration we are working hard to develop and drive broad acceptance of a single, unifying language. For example, we often hear people using the term 'validation', but this means vastly different things to clinical scientists, data scientists, and engineers. We cannot make progress until we are talking apples and apples.

Similarly, we need shared evidentiary frameworks to facilitate collaborative approaches to advancing the use of digital technologies such as algorithms. Until there is a shared definition of what good looks like and a way to document it, redundant work is going to occur.

Finally, we need to develop a 'bilingual' workforce capable of understanding the basics of terminology and best practice across clinical and technical practice."

- Jennifer Goldsack, Executive Director at Digital Medicine Society (DiMe)
– follow @goldsackjen and @_DiMeSociety

"Both technical experts (data scientists, data engineers, software engineers, …) and business domain experts (clinicians, auditors, root cause analysts, trial monitors/managers/overseers, ..) are needed and have a role to play in a successful AI transformation. In our team at Janssen, here are a few ideas that we have tried to help accelerate this transformation:

• The “test kitchen” is a diverse group representing business experts interested in analytics tools, that are given early access to tools in development, shape aspects of the tool and define/evangelize use cases once rolled out.
• Our integrated data science teams comprise domain experts who are upskilling in data science principles and data scientists who are learning the domain, working together to solve problems.
• Projects undertaken by such integrated teams (so-called “Integrated Quality Labs” or IQ-Labs) are kept nimble, rapidly prototyped and iterated until they prove their value in a Proof-of-Concept, at which point they may become a full-blown Project.
• Wholehearted use of “Enterprise Agile Tools” by all in the team, i.e. Atlassian JIRA for tracking work packages / ideas / features / bugs and lightweight project management, Atlassian BitBucket for version control of code, and AWS S3 for version control and storage of data.
• Regular upskilling seminars (“Come Learn with Us”) by both data scientists and domain experts reaching out to and educating a wide cross-section of the department in the fundamentals of AI and advances made / capabilities available internally. We want the larger department over time to become “informed consumers” of AI/ML, even if they are not directly involved in AI projects."

- Deepak Bandyopadhyay, Associate Director of Quality Analytics at Janssen

"Innovation and advancements in any sector often happen at the edge - where people from different backgrounds and experience come together to work on new things together. Different expertise can stimulate, challenge, and catalyse new thinking - leading to exciting new approaches. We should be encouraging clinical experts to work with leading technologists in a collaborative way to design the optimal trials of the future.

However, technology and clinical expertise is not sufficient to change the game. The melting pot must include patients, communications experts, and the designers of consumer-grade experiences. This is best achieved through open collaboration between stakeholders therefore shaping and innovating together."

- Bruce Hellman, CEO and Co-Founder of uMotif - follow @umotif