An AI reads structural heart disease off an ECG better than cardiologists can

An electrocardiogram costs a few dollars and takes a few minutes. An echocardiogram — the test that actually diagnoses structural heart disease — costs far more, needs a trained sonographer and a machine, and therefore gets ordered only for people a clinician already suspects. A model published this week in Nature is an attempt to close that gap by making the cheap test carry more of the diagnostic load [s1].

The result is one of the stronger AI-ECG papers to date, partly because of what the model does and partly because of what its authors chose to publish alongside it.

What was built

EchoNext is a convolutional neural network trained to predict the presence of structural heart disease from a 12-lead ECG trace plus seven standard values printed on the ECG: age, sex, atrial rate, ventricular rate, PR interval, QRS duration and corrected QT [s1].

The training data were 1,245,273 ECG–echocardiogram pairs from 230,318 adult patients, collected between December 2008 and 2022 at eight NewYork-Presbyterian affiliated hospitals [s1]. These were split by patient into training (149,819 patients, 796,816 pairs), validation (35,780) and test (44,719) sets, with only the most recent pair retained per patient in validation and test [s1].

"Structural heart disease" here is a composite label defined against echocardiography guidelines: left ventricular ejection fraction of 45% or less; maximum left ventricular wall thickness of 1.3 cm or more; moderate or severe right ventricular dysfunction; pulmonary hypertension; moderate or severe aortic stenosis, aortic regurgitation, mitral regurgitation or tricuspid regurgitation; or a moderate or large pericardial effusion [s1]. Prevalence of that composite in the test set was 36% [s1].

The composite is a deliberate design choice, and the authors explain it: precision depends on prevalence, so bundling conditions that share a diagnostic pathway — all of them confirmed by echocardiography — raises achievable precision at no extra operational cost, because a high score means "refer for echo" in every case [s1].

Performance

On the internal test set, EchoNext reached an area under the receiver operating characteristic curve of 85.2% (95% CI 84.5 to 85.9), an area under the precision-recall curve of 78.5% (95% CI 77.2 to 79.6) and a diagnostic odds ratio of 12.8 (95% CI 11.6 to 14.1) [s1].

Performance varied by component. Right ventricular dysfunction (AUROC 91%) and low left ventricular systolic function (90%) were detected best; low left ventricular wall thickness (77%), aortic regurgitation (78%) and pericardial effusion (80%) were detected worst [s1]. Across the eight hospitals AUROC ranged from 82% to 87%, and across clinical contexts from 79% to 84% [s1]. The authors report no clinically relevant differences by race or ethnicity or by sex, with slightly better discrimination in younger patients [s1].

External validation used three independent cohorts: Cedars-Sinai (10,177 patients), the Montreal Heart Institute (10,862) and UCSF (6,106) [s1]. Disease prevalence was much higher at those sites — 54%, 52% and 46% against 36% internally — and AUROC fell by 5 to 7 percentage points, to 78–80% [s1]. At a fixed sensitivity of 70%, positive predictive value was comparable but specificity dropped by 10 points [s1].

That drop is the honest headline for anyone thinking about deployment. A model built on one health system's population loses ground on another's, and the loss shows up in specificity — meaning more false positives, meaning more unnecessary echocardiograms.

Against cardiologists

Thirteen cardiologists reviewed a 150-ECG set drawn from the test data, with 41% structural heart disease prevalence and a mean patient age of 67.0 (SD 19.6), completing 3,200 reviews in total — 1,600 without the model's score and 1,600 with it [s1].

On that set, EchoNext achieved 77.3% accuracy, 72.6% sensitivity and 80.7% specificity [s1]. Unassisted, the cardiologists reached 64.0% accuracy (95% CI 61.6 to 66.4), with 61.1% sensitivity and 66.1% specificity [s1]. Shown the model's score, they improved to 69.2% accuracy (95% CI 66.9 to 71.4) — better, but still below the model alone [s1].

The authors state the limit of this comparison themselves: the task used only the ECG, without the clinical history, physical examination or other testing data a physician would normally have [s1]. It measures ECG interpretation, not clinical judgement.

One subtler finding is more interesting than the headline gap. Cardiologist accuracy differed between clinically normal and abnormal ECGs (69% versus 62%), while EchoNext scored 77% on both, despite structural heart disease prevalence differing sharply between those groups (25% versus 46.5%) [s1].

The prospective part, read carefully

The abstract describes prospective evaluation in a clinical trial of patients without previous cardiac imaging [s1]. The detail matters.

The DISCOVERY trial was a 100-patient open-label stratified-sampling prospective study, and patients were recruited on the basis of their score from ValveNet — an earlier model from the same investigators, trained to detect moderate or greater left-sided valvular disease [s1][s2]. ValveNet reported AUROC 0.88 for aortic stenosis, 0.77 for aortic regurgitation, 0.83 for mitral regurgitation and 0.84 for any of the three [s2].

Median age of DISCOVERY participants was 80 (IQR 72 to 86) and 43% were male [s1]. Among 53 patients with high ValveNet scores, 17% had moderate or greater left-sided valvular disease and 53% had structural heart disease; among 47 with moderate scores, 0% and 19% respectively [s1]. Both differences were significant (p=0.005 and p=0.003) [s1].

EchoNext itself was applied to those 100 ECGs retrospectively, after the trial finished [s1]. Stratified by EchoNext score, the high-risk group (n=33) had 24% left-sided valvular disease and 73% structural heart disease; moderate risk (n=50), 2% and 28%; low risk (n=17), 0% and 6% [s1].

So EchoNext has not been prospectively deployed as a recruitment tool. A related model was, and EchoNext was scored against the same patients afterwards.

Silent deployment

The larger real-world test was passive. EchoNext ran in the background on 124,027 ECGs from 84,875 patients with no previous echocardiogram, acquired between 1 January and 16 September 2023, none of whom appeared in training [s1]. Eighteen per cent (15,094) went on to have a first echocardiogram in routine care, and 38% of those (5,744) were newly diagnosed with structural heart disease [s1]. In that subset the model held up: AUROC 83%, AUPRC 81% [s1]. At the predefined score cutoff of 0.6, 27% were flagged high risk, giving positive predictive value 74% and sensitivity 53% [s1].

Among the 69,781 patients who never got an echocardiogram, 3,444 (5%) were flagged high risk [s1]. Their true disease status is unknown [s1]. The authors model what it might be — at an assumed 10% prevalence and 50% sensitivity, projected positive predictive value would be 46.5%, implying roughly 1,998 findable cases [s1] — but that is a projection resting on an assumed prevalence, not a measurement.

The release

Alongside the paper, the authors released model weights and a de-identified annotated dataset of 100,000 ECGs from 36,286 Columbia patients, labelled for structural heart disease and its components from matched echocardiograms, with dates shifted per patient by more than a year to preserve intervals [s1]. A model trained only on that public subset reached AUROC 82.0% (95% CI 80.9 to 83.0) against 83.1% for the full multicentre EchoNext on the same test set [s1].

Most AI-ECG models and their training data are proprietary, which the authors name as a barrier to comparison and to evaluating generalisability [s1]. Releasing a benchmark that gets within about a point of the flagship model is the part of this paper most likely to matter in three years.

What it does not settle

Nothing here is a clinical outcome. The model predicts echocardiographic findings, and echocardiographic findings are not the same as benefit from being found. Whether flagging 5% of ECGs for echocardiography reduces mortality, or mainly generates downstream testing, is a question this design cannot answer.

The external-validation drop, the reliance on a related model for the prospective arm, and the unknown status of the 69,781 unimaged patients are the three places where enthusiasm should slow down.