Seven gut-microbiome tests got identical stool. They disagreed as much as different people

Researchers at the US National Institute of Standards and Technology took a single homogenised batch of human faecal material, divided it up, and sent it to seven direct-to-consumer gut microbiome testing companies as if they were seven ordinary customers ordering three kits each [s1]. The results came back so different from one another that the variability between providers was on the same scale as the biological variability between different human donors [s1].

Put plainly: which company you send your sample to can matter as much as whose sample it is.

How the test was run

NIST has been developing standardised human faecal reference materials — homogeneous and stable batches confirmed by multi-omic analysis — precisely so that microbiome measurement workflows can be checked against something that does not vary [s1]. For this study, three kits were ordered from each of seven companies through the companies' own websites, giving 21 collection kits [s1]. Each was filled from the same homogenised pool, packed according to each company's instructions, and returned [s1].

None of the shipping packages included ice, so every sample travelled at ambient temperature; the study was run in the spring [s1]. Reports came back between two and eight weeks after shipping [s1]. The companies were not told the assessment was happening until after all samples had been processed and every final report was in hand [s1].

What came back

The clearest single number is the species count. Given the same faecal material, the seven companies reported, across their three replicates: 66, 72 and 121 species; 20, 29 and 42; 615, 713 and 767; 96, 98 and 99; 139, 163 and 163; 1,471, 1,626 and 1,701; and, for a company that returned two replicates, 259 and 259 [s1].

Sequencing depth did not explain the spread. One company reported over 600 species per replicate from read depths of 23.7 to 29.4 thousand reads, while another reported 96, 98 and 99 species from replicates of 11 to 15 million reads [s1]. Reporting thresholds differed too: cut-offs for including a taxon in the report ranged from 0.1% down to 0.000008% [s1].

One inconsistency inside the paper is worth recording. The methods table gives that company's three replicates as 96, 98 and 99 species, while the discussion section describes the same three replicates as 95, 98 and 99 [s1]. The discrepancy does not change the finding, but the table and the narrative do not match.

Methods differed at every stage. Some companies asked customers to swipe used toilet paper; others supplied a proprietary collector to be inserted into stool in ten to twelve places [s1]. Some supplied a storage buffer, some shipped the swab dry [s1]. Three used 16S rRNA gene amplicon sequencing — two of the V4 region, one of V3-4 — and four used whole metagenome shotgun sequencing [s1].

The formal comparison is the part that matters most. The researchers fitted a Bayesian model to the 18 genera that every company detected in at least one replicate, comparing variation caused by different methods on the same sample against variation caused by the same method on eight different donors [s1]. Only one genus, Haemophilus, had a credible interval falling entirely below 1 — meaning less variation between methods than between people [s1]. For the other 17, the interval either sat entirely above 1 or straddled it [s1].

The interpretation problem sits on top of the measurement problem

Even a perfectly reproducible measurement would have to be interpreted against something. The study found no standard practice among the companies for choosing a comparison "healthy" population [s1]. Some used data generated in-house, either from a cohort recruited for the purpose or aggregated from previous customers' samples, with little information given about the demographics of those groups [s1]. Others compared against pre-existing public datasets such as American Gut or the Human Microbiome Project — which, the authors point out, is undermined by the same methodological variability, since differences from a comparison set generated by a different protocol may reflect biology, methodology, or methodology alone [s1].

The authors also note that there are currently no regulatory-approved clinical microbiome diagnostic tests in the US, and only one sequencing-based test carrying the CE-IVD designation in Europe [s1]. These products sit between strictly regulated medical devices and minimally regulated general health and wellness products — a distinction the authors say may not be readily apparent to consumers [s1].

What the clinical field says

An international multidisciplinary expert panel convened to standardise best practice for microbiome testing reached a compatible conclusion from the clinical side. Their consensus statement, published in The Lancet Gastroenterology & Hepatology, states that evidence supporting the clinical usefulness of the gut microbiome as a diagnostic tool is scarce, and that an increasing number of commercial providers offer direct-to-consumer tests with no consensus on regulation and no proven value in clinical practice [s2].

The panel's stated concern is not merely wasted money. It warns of considerable waste of individual and health-care resources and potential drawbacks in the clinical management of patients [s2].

The specific harm the authors worry about

The NIST authors make the point that dietary recommendations may seem innocuous, but that a significant proportion of people who buy these tests are people with chronic gut conditions who have struggled to find effective treatment [s1]. For them, variable results or recommendations that do not work risk not only a loss of faith in the science but a delay in appropriate medical care [s1]. The authors also note that some companies recommended customers start taking costly supplements — probiotics among them — sold by the same company, and for which there is very little clinical evidence of efficacy [s1].

The limits of this study

The reference material is not ground truth. The authors are explicit that it does not establish which company's answer is closest to the sample's actual composition, and that determining this was not the study's goal [s1]. What a homogeneous standard can establish is precision — whether a workflow gives the same answer twice — and the finding there is mixed rather than uniform. Reproducibility within a single company's locked-down workflow was generally very good; the failure is between companies, not within them [s1].

That distinction is itself useful for a reader. A repeat test from the same company will probably agree with itself. It is the leap from that number to a statement about your gut health, and the comparison against a "healthy" reference of unstated provenance, that the evidence does not currently support.

Sources

• Evaluating the analytical performance of direct-to-consumer gut microbiome testing services — Communications Biology, 2026-02-26
• International consensus statement on microbiome testing in clinical practice — The Lancet Gastroenterology & Hepatology, 2024-12-05