"AI clinical decision support has entered primary care" has been one of the loudest narratives in health informatics over the past few years. But there is an awkward fact: no systematic review has yet assessed the real-world impact of these systems on clinical decision-making and patient outcomes. On October 8, 2026, a team from Tsinghua University published such a review in the Journal of Medical Internet Research (JMIR)—a synthesis focused specifically on real-world evaluations of AI clinical decision support systems (CDSS) used by clinicians in primary and outpatient settings. The review examined evidence across four outcome domains: diagnostic performance, care modification, clinician efficiency, and patient-important outcomes. The authors’ one-word summary of the findings is "mixed." Breaking down this word reveals three directly actionable insights for procurement, clinical practice, and research.
## First, the screening method: a very fine sieve
The methodology of this review deserves attention first—how robust the conclusions are depends on how rigorously the literature was screened. The three authors are from the Department of General Practice at Beijing Tsinghua Changgung Hospital. Following the PRISMA 2020 guidelines, they searched five major bibliographic databases—PubMed, Scopus, Web of Science Core Collection, Cochrane CENTRAL, and CINAHL Ultimate—plus one supplementary platform (EBSCOhost) and two trial registries, covering literature published between 2011 and 2025. Eligibility criteria were narrowly defined: studies had to evaluate either learning-based (data-driven) or use-case–based AI components; must involve clinicians using the system routinely in primary or outpatient settings; and must report outcomes related to clinical, behavioral, process, efficiency, or diagnostic accuracy measures. Screening was conducted independently by two reviewers; risk of bias was assessed using tools appropriate to each study design; and GRADE was applied to rate evidence quality across the four outcome domains.
The result: 66 full-text articles underwent assessment, and 24 studies met the review’s inclusion criteria. Then came the critical step—of these 24, only 10 employed genuinely AI-based components and thus constituted the population under evaluation; the remaining 14 were deterministic rule-based tools, retained by the review authors solely as contextual comparators. The authors’ conclusion applies to evidence published “up to December 2025.”
## First finding: Most systems labeled "AI" are not AI
10 versus 14—that ratio itself is a finding. A majority of the literature on "AI decision support" in primary care actually describes deterministic rule-based tools, i.e., traditional if-then logic engines (to clarify: this article’s translation notes that the review distinguishes the two tool types strictly methodologically—not to accuse any party of “masquerading”). Learning-based or use-case–based AI qualified as the target evaluation population; rule-based tools were retained only for contextual comparison—the review draws this line with exceptional clarity via its inclusion criteria.
This line should also be drawn into hospital procurement questionnaires—and as the very first question: Is the CDSS you intend to purchase learning-based or rule-based? These are not the same class of technology, and their evidence bases cannot be conflated. If a vendor responds only with “uses AI technology,” the question remains unanswered.
## Second finding: Evidence temperature cools progressively across the four outcome domains
The review’s results are laid out across the four outcome domains, revealing a clear gradient—this is the article’s interpretive restatement of the review’s findings; the authors themselves used the term “mixed.”
The warmest domain is detection and diagnosis—but “warm” here means only that positive signals exist; internally, this domain itself is highly inconsistent. AI-enabled ECG analysis improved diagnosis of newly detected low ejection fraction (odds ratio [OR] 1.32, 95% CI 1.01–1.61; lower bound barely touches 1, marginal significance reported verbatim); yet, in the same task—“mining diagnoses from existing data”—machine learning–derived dementia biomarkers extracted from electronic health records failed to increase dementia diagnosis when used alone (adjusted OR 0.84, 95% CI 0.63–1.11). Discrepancies among diagnostic accuracy studies are similarly large: AUROC for melanoma classification reached 0.960, whereas for glaucoma it was only 0.80, with sensitivity at 65%.
The middle domain—changes in care processes—features heterogeneous study designs: non-randomized before-and-after studies reported urinary tract infection treatment success rising from 75% to 80%; similar studies reported that 3 of 4 general practitioners changed diabetic retinopathy screening criteria under AI influence; a pre-exposure prophylaxis (PrEP) trial showed overall ineffectiveness; and a multi-component fall-prevention CDSS improved shared decision-making but yielded uncertain effects on medication changes.
Next comes efficiency: In asthma care, clinicians self-reported an estimated reduction in chart review time—note the two qualifiers: “self-reported” and “estimated,” not objectively timed measurements.
The coldest domain is patient-important outcomes: Of the two trials assessing patient-important outcomes, neither demonstrated benefit attributable to AI-CDSS; one even found musculoskeletal function outcomes favored usual care.
## The authors’ own concluding remarks
The authors’ conclusions have three layers: (1) Evidence published up to December 2025 shows mixed effects of AI-CDSS in primary care—some studies report improvements in detection or care processes, but patient-important benefits remain unproven; (2) Heterogeneity and limitations in outcome measurement prevent causal interpretation of these differences; and (3) Future trials should concurrently assess both clinical decision-making and patient outcomes. The third point constitutes a design-level prescription for the entire field. The authors declared no conflicts of interest.
## My three judgments
The following are integrated interpretations derived from reading this review alongside the detailed design features of its included studies—not direct conclusions stated in the original text.
First, “learning-based or rule-based?” should become the field’s foundational question. Most literature labeled “AI” actually describes rule engines—meaning that any aggregate assessment of “AI decision support effectiveness” that fails to distinguish the two technologies is fundamentally conflating accounts. Procurement, evaluation, and regulatory review all begin with peeling apart the label.
Second, evidence temperature declines along the causal chain: the closer the outcome is to what machines can directly alter (detection, clinician behavior, self-reported efficiency), the warmer the evidence; the closer it is to outcomes patients truly experience, the colder the evidence—so cold that neither of the two trials demonstrated benefit. This gap between “increased detection” and “patient benefit” aligns consistently with patterns observed across screening-oriented AI applications (a cross-domain observation made in this article): detecting more cases does not equate to improving disease course.
Third, the field’s marginal contribution has shifted from “yet another single-disease study” toward trial design. The review’s prescription is highly specific—simultaneous assessment of decision-making and outcomes—and its GRADE-based four-domain framework (diagnostic performance, care modification, efficiency, patient outcomes) is itself a ready-made evidence self-audit checklist: vendors can use it for internal assessment; payers can use it to set reimbursement conditions; regulators can use it to build evaluation frameworks.
## What you can take away
If you work in a primary care system or hospital: Write outcome monitoring into your CDSS contract—if a vendor can only demonstrate detection rates and clinician satisfaction, what you’re buying may be a workflow tool, not a clinical tool. First ask whether it’s learning-based or rule-based, then ask for evidence. If you are a vendor: the four outcome domains constitute your self-audit checklist; the current state—“patient-important outcomes remain unproven”—cannot be overcome by more retrospective detection studies. If you are a clinician: When AI prompts you to change screening thresholds (think of those 3 of 4 general practitioners), pause and ask about the evidence grade—behavioral change itself is not the goal. If you are a payer: When including AI-CDSS in reimbursement schedules, require evidence at the outcome level or post-deployment monitoring clauses; detection rates and clinician satisfaction alone must not serve as standalone payment justifications. If you are a researcher: The field’s most pressing gap is no longer another single-disease accuracy study—it lies in “decision + outcome co-assessment” trial designs; this review’s eligibility criteria can be directly adopted.
If the community hospital near your home is procuring an “AI decision support system,” what would you want the first question asked to be—does it detect accurately, or does it make patients better off? Join the discussion in the comments.
Source: Journal of Medical Internet Research, October 8, 2026 (PMID 42849036, https://doi.org/10.2196/98630).