A Sydney hospital's AI predicts who gets admitted. Vital signs turned out not to matter

An Australian emergency department prediction model published this year does something most clinical machine-learning papers do not: it reports, in detail, which inputs turned out to be useless.

The tool is called START-AI — the Sydney Triage to Admission Risk Tool with Artificial Intelligence. It is an extension of an existing clinical score, START, and its job is narrow: at the point of triage and shortly after, estimate whether an adult who has just walked into the emergency department will end up admitted to an inpatient ward [s1]. That single binary prediction is what drives bed planning, and getting it wrong in bulk is how a department ends up with patients waiting in corridors.

What was built, and on what

The development study drew on adult presentations — age 16 and over — at a single inner-city tertiary referral hospital emergency department between 1 January 2023 and 30 June 2025 [s1]. Patients who were dead on arrival or who left before completing treatment were excluded [s1]. That left 162,915 cases, of which 27.31% ended in an inpatient admission [s1].

The architecture is a stack rather than a single model. The original START score was combined with vital signs, blood test results and CT imaging orders using a gradient-boosted decision tree algorithm (XGBoost), alongside a pre-trained transformer model that reads clinical free text [s1]. The free text is the notable part: triage comments and case history notes are unstructured prose written under time pressure, and most deployed hospital risk scores cannot use them at all.

The final stacked model reached an area under the receiver operating characteristic curve of 0.88 (95% CI 0.88–0.89), with weighted accuracy of 0.84 (95% CI 0.84–0.85) and an F1 score of 0.83 (95% CI 0.83–0.84) on the held-out test data [s1]. A sensitivity analysis that reclassified inpatient stays shorter than 24 hours as potential discharges — a fairer test, since a sub-day admission is arguably not the outcome the tool is meant to flag — produced an AUROC of 0.89 [s1].

The calibration caveat the authors flag themselves

Discrimination is not the same as calibration. A model can rank patients correctly while still being systematically wrong about the absolute probability it assigns. START-AI's calibration was reported as adequate, with an R² of 0.92 (95% CI 0.67–0.99), but with an explicit drop-off in correlation at predicted probabilities above 0.80 [s1].

That upper range is exactly where a bed manager would want the number to be trustworthy. A patient the model says is 90% likely to be admitted is the one you would start arranging a bed for. The authors do not paper over this: their stated conclusion is that prototype development and prospective evaluation are still required to assess how the model performs in a live clinical setting [s1]. As of the development paper, this is a retrospective model on historical records, not a deployed tool with outcomes attached.

What the explainability analysis found

The follow-up paper, published in July, re-ran the model with each feature added sequentially to see what each one was actually contributing [s2]. It used a slightly different two-year window of single-centre electronic medical record data, and reported the original START tool alone at an AUROC of 0.78 (95% CI 0.77–0.78), rising to 0.90 (95% CI 0.89–0.90) once all START-AI features were added [s2]. The gap between that 0.90 and the 0.88 in the development paper reflects the different data windows, not a change to the model — worth noting before anyone quotes a single headline figure.

The features associated with a stepwise increase in cumulative AUROC were triage comments, emergency department case history notes, any blood test result — lactate and C-reactive protein in particular — and any CT order [s2]. On overall model importance, the highest-ranking features were the presence of any blood test result, the START score itself, and C-reactive protein [s2].

And vital signs? They did not appear to be associated with stepwise increases in AUROC [s2].

Why that null result is the interesting part

Heart rate, blood pressure, respiratory rate and temperature are the measurements an emergency department takes on essentially everyone, immediately, and they anchor most existing triage instruments. Finding that they added nothing measurable on top of the rest of the model is a genuinely useful negative finding, and the fact that it survived into print rather than being quietly dropped is worth something on its own.

There is a plausible mechanical explanation, though the papers do not test it: the presence of a blood test result being the single most important feature suggests the model is partly learning what clinicians already decided. A clinician who orders bloods and a CT has already formed a view that this patient is sick. The model may be reading that judgement rather than independently discovering it. That does not make the prediction less accurate, but it changes what the tool is — less an early-warning system that sees something staff missed, more a formalisation of a decision already made, available earlier and at scale.

What to watch

The authors of both papers are explicit that the work is not finished: the development paper calls for prospective evaluation [s1], and the explainability paper frames its results as informing how the tool can be further developed and deployed [s2]. Both are single-centre. Neither reports whether using the model changes any patient outcome, length of stay, or emergency department flow measure — because neither study was designed to, and no deployment has been reported.

The measure that will matter is whether a prospective evaluation at a second Australian site reproduces the 0.88–0.90 discrimination, and whether the calibration weakness above 0.80 holds up or resolves. Until then, this is a well-characterised retrospective model with an unusually honest account of its own internals.

Sources

• [s1] The Sydney Triage to Admission Risk Tool With Artificial Intelligence (START-AI): Prediction of Inpatient Admission From Emergency Departments Using Ensemble Machine Learning, Emergency Medicine Australasia, 8 March 2026. https://doi.org/10.1111/1742-6723.70240
• [s2] The Sydney Triage to Admission Risk Tool With Artificial Intelligence (START-AI) to Support Decision Making in Emergency Departments: Model Explainability and Feature Importance Analysis, Emergency Medicine Australasia, 28 July 2026. https://doi.org/10.1111/1742-6723.70285