Use and principles of automated ECG measurements in clinical trials

by Dr. Tim Callahan, Biomedical Systems

Even before the approval of the ICH E14, automated electrocardiographic (ECG) measurements and interpretations were available from the ECG machine manufacturers. These measurements and interpretations appear on the printed ECG. Each manufacturer advised caution, however, when using these readings. The final arbiter for ECG measurements and interpretation, and thus the gold-standard has always been the cardiologist.

In the era of the E14, the number of ECGs collected in early phase drug development has multiplied exponentially. Previously, the standard was to collect only safety ECGs at the Phase 1 unit. However, a simple crossover Thorough QT (TQT) trial can consist of 4-periods, more than 40 subjects, multiple time-points, with ECGs collected in triplicate. This can lead to approximately 10,000 ECGs. It has also become a regular practice for the sponsor to collect multiple ECGs in the early development (FIM, SAD, MAD, etc.) to give some indication as to the drugs’ effect on the ECG measurements.

The large number of ECGs now collected in the short time-course of a Phase 1 study places a great burden for the reader to read all of the ECGs in a timely manner. The E14, therefore, allows for more than one person to read the ECGs from a TQT trial as long as each subject is assigned to a single “skilled reader”. Because of inter-reader variability concerns, it is best to keep the number of readers to a minimum. The effect is to reduce the time to database lock, but it can be argued that there is an alternative. This would be to use automated readings.

The rest of this paper will detail how automated readings are obtained, when they can be considered, types of automated readings, how the readings can differ between algorithms, how to check for quality, and the best practices when using automated readings.

Manufacturers of Automated Algorithms

Automated algorithms come from two main sources; ECG machine manufacturers and third-party vendors. Manufacturers such as GE, Mortara, Phillips, and Schiller can have the ECG measurements and interpretation printed on the unconfirmed copy of the ECG (as well as residing in the database).

Newer, third-party algorithms (from companies such as AMPS, OBS, Monebo, and NewCardio) can measure digital ECGs (usually in an XML format) after collection. Because the algorithm is not connected to an ECG machine, these programs usually process the data in a batch format, probably at the end of the trial. Therefore, the readings are not available at collection time.

Automated Measurements and the ICH E14

Regarding the measurement of the ECG, the ICH E14 states:

“At present, this would usually involve the measurement by a few skilled readers (whether or not assisted by computer) operating from a centralized ECG laboratory. If well-characterized data validating the use of fully-automated technologies become available, the recommendations in the guidance for the measurement of ECG intervals could be modified.”

Currently, automated measurements are being used in some situations (as described below). Regardless of the method used to measure the ECG, the sponsor is responsible for the data. Therefore, regardless of the measurement methodology, the sponsor has to be comfortable with and confident in the final database.

Why Use Automated Measurements

In the ICH E14 era traditional ECG measurements are made by cardiologists who placed electronic calipers (or annotations) on the fiducial points of the electronic ECG. Intervals are then calculated and stored in the computerized system. The process of measuring an ECG takes a discrete amount of time with the variables being the technical quality of the ECG, number of annotations to be placed, and the ability to discriminate the onsets and offsets of the ECG waveforms. Depending on the number of ECGs in a clinical trial (which can be upwards of 10,000 or more), the time it takes to measure all of the ECGs can be considerable.

With the time constraints of the clinical trial database lock, more than one reader is needed. As can be seen with the quote from the ICH E14 above, a few skilled readers may be utilized. However, because readers have their own opinions as to where the fiducial points are located on the ECG, and consistency is important, inter- and intra-reader variability needs to be addressed. ECG core laboratories have dedicated significant resources to benchmark and measure inter- and intra-reader variability.

Finally, ECGs in TQT trials are usually collected in triplicate. That is, three ECGs are collected within approximately a 5-minute window at a nominal time point relative to dosing. The triplicate ECGs are used to reduce the normal physiologic variability inherent in ECG measurements. The standard deviation of the QTcF is often used as a measure of data quality. Manufacturers of automated algorithms often tout the fact they can help lower the standard deviation of the QTcF within the triplicate ECG, thus producing a higher-quality database.

With these thoughts in mind, the fully-automated approach has been shown to be effective in many cases. This is because the time to a locked database can be reduced as the computer takes a much shorter time per ECG as compared to a human reader. Another benefit is that the computer will have little to no intra-reader variability. The result is that the consistency of the readings is of a high quality.

How Automated Measurements Work

In general, automated measurements come from a representative beat based on all 10-seconds of the digitally-acquired ECG. For example, Mortara Instruments makes automated measurements on a global complex.

The process begins by taking each lead of the ECG and creating a “median” complex from a primary, or dominant normal beat. All of the beats with the primary morphology are used for the creation of the median beat. After medians are created for each lead, a global median is created by aligning the individual median beat. From there, the earliest onset to the latest offset is measured for all variables (PR, QRS, QTand QT). The result of using the earliest onset to the latest offset will be to have a longer QT-interval measurement than the human reading, in many cases. Heart rates for automated algorithms are usually calculated over the entire 10-second ECG.

Not All Algorithms Work the Same

Kligfield, et al. published a comparison of the performance of two algorithms from two different manufacturers. The authors collected 2 sets of ECGs (one for each manufacturer’s digital electrocardiograph) at the same time. The authors’ note that the acquisition was not simultaneous due to the prerecording processing operations differs between the ECG machines.

Not only did the algorithms measure the QT interval differently, automated measurements of the QT interval differed from one version to another within the same algorithm family by up to 24 milliseconds.

This study points out the importance of standardizing the equipment, including software versions, between all of the sites and subjects if automated readings are to be used from the ECG machine.

Best Practices

Automated algorithms are being used in many clinical trials. They can be a cost- and time-effective alternative to human readings under the right circumstances. Two issues need to be thought-through when considering using automated algorithms; ECG quality and adjudication of substandard readings.

Poor ECG quality will affect the accuracy of ECG measurements, whether by a human reader or an automated reading. High-frequency is the noise at the baseline level that interferes with the ability to determine, with precision, where one feature of the ECG ends and another begins. Unfortunately, there is no adequate solution to the problem once the ECG has been collected as ECG filters can slightly alter the morphology. The only adequate solution is to collect good quality ECGs.

Since not all ECGs are of the highest quality, and sometimes the computer reading fails, using an adjudication algorithm should be part of the clinical trial when automated algorithms are used. Adjudication should be considered based on ECG values that are not in the mainstream of the normal QT/RR relationship (so-called outliers) and when certain quality control metrics are not met by triplicate ECGs.

Finding outliers is in the dataset is not difficult. These values fall outside either the pre-defined value (such as a QTc of greater than 450 ms or a change from baseline of 30 ms) or a pre-defined relative value (such as a value greater than 2 standard deviations from the mean).

Another set of issues with the data quality arise when values appear normal but are not accurate. These values, which we term “inliers” here, can affect the overall quality of the database, negatively. To find these values, we must look at metrics such as within-triplicate standard deviation, relative changes from baseline as compared the other subjects, and within-triplicate ranges (to name a few). By running data quality metrics, database quality can be assured, regardless of reading methodology.

Discussion and Conclusions

Relatively recently, the The FDA was not in favor of automated measurements of ECGs until relatively recently.. Now, however, the FDA seems to be accepting of them with the caveat that the sponsor is responsible for the data.

It is widely recognized that automated measurements are best used with ECGs from normal, healthy volunteers where the ECG intervals are easy to read. This generally means the automated measurements are best used in early phase clinical trials. It is fortuitous that these trials are mainly where data is collected digitally and can therefore be read with automated algorithms.

Automated algorithms can be found from both third-party vendors and from the ECG machine manufacturers. Both types have been used in clinical trials. Regardless of what algorithm is used, standardization is important to make sure all ECG are measured with the same version of software and an adjudication algorithm is employed.