An LLM halved emergency discharge note time at a Korean hospital. Editing cost accuracy

Documentation is where most hospital AI deployments are actually happening — not diagnosis, not triage, but the writing. Two studies published three days apart in October measure what that produces, and both land on the same structural finding: the human editing step changes the output in more than one direction.

The Korean deployment study

Researchers at a 2,400-bed tertiary care hospital in South Korea developed and evaluated a large language model assistant for emergency department discharge notes [s1].

The model was a commercially available text generation transformer, fine-tuned on 592 representative cases selected by emergency physicians from ED visits between 1 September 2022 and 31 August 2023, with 50 cases held out for validation [s1]. Two processing pipelines were implemented depending on input data: one for patients managed solely by emergency physicians, using the ED initial record and prescription list, and one for patients requiring specialty consultation, using the ED initial record and the consultation request form [s1].

The validation was a within-physician comparison. Six emergency physicians first wrote discharge notes manually, then — after a one-hour washout — edited LLM-generated drafts [s1]. Three independent physicians evaluated 300 note sets, each containing a manual note, an LLM draft, and an LLM-assisted note, scoring them on completeness, correctness, conciseness and clinical utility on a 1–5 Likert scale [s1].

The 50 test cases had a mean patient age of 57.7 years (SD 23.1), and 28 of 50 patients (56%) were female [s1].

The two comparisons

Against manual notes, LLM-assisted notes scored higher on every dimension [s1]:

• Completeness: 4.23 (95% CI, 4.17–4.28) vs 4.03 (3.96–4.09)
• Correctness: 4.38 (4.33–4.42) vs 4.20 (4.14–4.26)
• Conciseness: 4.23 (4.18–4.28) vs 4.11 (4.05–4.17)
• Clinical utility: 4.17 (4.11–4.23) vs 3.85 (3.78–3.91)

All four differences were significant at P < .001 [s1]. Median documentation time fell from 69.5 seconds (95% CI, 65.5–78.0) to 32.0 seconds (29.5–36.0), also P < .001 [s1].

Against the unedited LLM drafts, the picture inverts on two of four measures. Edited notes were more concise (4.23 vs 3.98, P < .001) and equivalent in clinical utility (4.17 vs 4.16, P > .99), but scored lower in completeness (4.23 vs 4.34, P = .001) and correctness (4.38 vs 4.45, P < .001) [s1].

What the inversion means

The raw model draft was, on the study's own metrics, the most complete and most correct document produced in the experiment [s1].

Two readings are available and the study cannot distinguish them. One: physicians editing under time pressure removed accurate content, and the edit degraded the note. Two: the evaluators' definitions of completeness and correctness reward inclusion of detail that physicians deliberately cut as clinically irrelevant — in which case the edited note is better for its actual purpose and worse on the scale.

The conciseness result points toward the second reading. Physicians cut length, gained conciseness, and lost points on completeness. Whether that trade favours the patient reading the note is not something a Likert score answers.

Either way, the finding complicates the standard assumption that a human review step monotonically improves AI output. It does not, at least not on every axis at once.

The translation study

A study published on 24 October tests the same human-in-the-loop question in a different documentation task: translating free-text inpatient discharge instructions for patients whose language is not English [s2].

Translations into Arabic, Armenian, Bengali, simplified Chinese, Somali and Spanish were produced three ways — by ChatGPT-4o, by professional linguists, and by a human-in-the-loop process in which AI output was post-edited by professional linguists [s2]. Linguists, clinicians and family caregivers evaluated each on 1–5 Likert scales for linguistic and clinical characteristics [s2].

ChatGPT-4o alone performed variably against professional translation, with its poorest ratings in digitally underrepresented languages — Armenian and Somali [s2]. That is the expected failure mode: performance tracks training corpus size, and languages with a smaller web presence are worse served.

Human-in-the-loop translations, by contrast, achieved comparable and often better outcomes than professional translation across all languages. In Armenian, mean overall quality was 3.9 (95% CI 3.7–4.2) against 3.6 (3.4–3.9) for professional translation, p = 0.01 [s2]. Human-in-the-loop output was most frequently preferred (46.5% vs 28.4%) and took less time: mean 7.1 minutes (5.4–8.8) against 16.8 (13.7–19.9), p < 0.001 [s2].

The authors conclude that human-in-the-loop strategies may enable safe, efficient and equitable machine translation in clinical practice [s2].

Reading the pair

In the translation study, the human step rescued the weakest cases — it is what made Armenian and Somali output usable [s2]. In the Korean study, the human step improved on unaided writing but cost measured accuracy relative to the raw draft [s1].

The reconciling factor is probably the expertise mismatch. Professional linguists post-editing translations are experts in the exact task being edited. Emergency physicians editing clinical notes under a 32-second median are experts in medicine performing a secondary task quickly.

That suggests the design question for documentation AI is not whether to keep a human in the loop, but what that human's editing time is being spent on, and whether the interface directs it at the parts of the draft where errors are consequential.

Limits

The Korean study involved six physicians at a single hospital, evaluating 50 test cases [s1]. Six raters is a small base for generalising, the fine-tuning corpus was hand-selected by the same institution [s1], and a one-hour washout between writing manually and editing a draft on the same case is short — carryover is plausible.

Neither study measured patient outcomes. Note quality scores are not comprehension, adherence, or return visits. The translation study evaluated quality ratings from linguists, clinicians and caregivers [s2], not whether patients understood their instructions.

Both studies also evaluated documentation in controlled conditions rather than measuring what happens when the system is running live, under real ED load, on cases outside the curated set.

What to watch

The measurable endpoint that matters is downstream: unplanned return visits, medication errors after discharge, and comprehension in patients who received AI-assisted instructions. Time saved and Likert scores are proxies for those, and no study yet has connected them.

Sources

• [s1] Large Language Model Assistant for Emergency Department Discharge Documentation — JAMA Network Open, published 21 October 2025. https://doi.org/10.1001/jamanetworkopen.2025.38427
• [s2] Evaluating human-in-the-loop strategies for artificial intelligence-enabled translation of patient discharge instructions: a multidisciplinary analysis — npj Digital Medicine, published 24 October 2025. https://doi.org/10.1038/s41746-025-02055-6