When the words "antinuclear antibody (ANA) positive" appear on a blood test report, most people need not worry—ANA positivity rates in general U.S. population studies range as high as 14% to 27%, yet its positive predictive value for systemic autoimmune diseases (e.g., lupus, systemic sclerosis) is only about 11%: among 100 ANA-positive individuals, roughly 11 will ultimately receive a confirmed diagnosis. But for that small minority truly progressing toward disease onset, this lab result initiates a race against time—and until now, medicine lacked any evidence-based rule to identify who should be fast-tracked. Referral decisions rested solely on individual clinicians’ judgment.
A randomized trial published November 6, 2024, in NEJM AI provided, for the first time, an answer grounded in real-world clinical workflow: integrating an AI risk model into the electronic health record and conducting a two-year randomized controlled trial yielded a median time from ANA positivity to confirmed diagnosis of 43 days in the intervention group versus 138 days in the control group. The numbers are striking—but what the numbers omit deserves even closer scrutiny.
## A Long-Standing Problem Without Tools
The root lies in the mismatch between overuse and scarcity. The American Board of Internal Medicine and the American College of Rheumatology’s “Choosing Wisely” campaign has long advised: do not order ANA testing without specific symptoms. Yet in reality, up to 22% of rheumatology referrals stem solely from an ANA-positive result; and among those referred, only 11% to 20% ultimately receive a diagnosis of systemic autoimmune disease. Rheumatologists are already in short supply; unnecessary referrals crowd appointment slots, pushing genuinely urgent cases to the back of the queue—lengthening wait times. And at the time, no validated, clinically deployable tool existed to help physicians interpret what an ANA-positive result actually meant.
In March 2024, the same research team published their solution in Frontiers in Immunology: they randomly sampled 2,000 ANA-positive individuals from de-identified electronic health records of over 3.6 million patients at an academic medical center, manually reviewed each chart to confirm whether a rheumatologist later diagnosed systemic autoimmune disease—the diagnosis was defined by rheumatologist judgment as the gold standard, not algorithmic self-assertion. Using readily available variables—including age, sex, ANA titer, platelet count, disease-specific autoantibodies, and billing codes associated with symptoms—they built an **interpretable scoring model**: essentially a weighted summation score sheet, with human-readable weights for each item. In the training set (1,030 patients), the area under the curve (AUC) was 0.83 (95% CI, 0.79–0.86); in the validation set (449 patients), it was 0.75 (0.68–0.81). AUC measures a model’s ability to separate future cases from non-cases: 0.5 equals random guessing; 1.0 equals perfect discrimination. The authors stated their intent plainly: expedite referral for high-risk individuals, reassure low-risk ones, and reduce unnecessary referrals.
## Embedding the Model into Real Workflow—Then Randomizing
What follows is the rarity in this story: they did not stop at AUC. The model was integrated into the electronic health record system, followed by a two-year **randomized pragmatic trial**—“pragmatic” meaning it did not alter the framework of usual care but evaluated impact within authentic clinical workflow. Such trials assess not “how accurate the model is on paper,” but “does deploying it and acting on it actually improve patient care?” All newly identified ANA-positive patients were randomized 1:1, and **randomization was independent of risk score**: in the intervention group, those classified as high-risk by the model were invited to undergo accelerated rheumatology evaluation; in the control group, risk scores were calculated identically but triggered no action. Both groups received standard care, and anyone could be referred through normal channels—per third-party trial registry documentation, this was an AI-assisted process: the model assigned scores and issued invitations only; every diagnosis remained the rheumatologist’s decision. Researchers reviewed charts every six months to verify who completed rheumatology evaluation and who received a diagnosis.
A total of 2,385 individuals participated in randomization. After excluding those excluded post-randomization (approximately 425 individuals, ~18%, per abstract figures—reasons for exclusion were not detailed in the abstract, and the third-party registry also noted this limitation), the final analysis included 954 in the intervention group and 1,006 in the control group. The primary endpoint was “time from ANA positivity to confirmed diagnosis”: the intervention group showed significantly shorter time, with a **hazard ratio of 1.9 (95% CI, 1.3–2.7; P < 0.001)**—roughly, the probability of reaching diagnosis within any given time interval nearly doubled; among those ultimately diagnosed, the median time to diagnosis was **43 days in the intervention group versus 138 days in the control group**.
## What the Numbers Do Not Say
Now consider what this trial did *not* demonstrate. First, the primary endpoint is **time to diagnosis**—a care-pathway metric. Faster diagnosis is inherently beneficial; no one welcomes months of anxious waiting. Yet the trial provides *no evidence* that earlier diagnosis alters disease course, organ damage, or treatment response. “Faster” is a starting point—not the finish line. Second, secondary endpoints were not met: among those who underwent rheumatology evaluation within six months, 23% in the intervention group were diagnosed versus 16% in the control group (P = 0.07), **not statistically significant**. This endpoint was intended to address “did the model select the right people?” Its lack of significance means we cannot disentangle how much benefit came from “accurate selection” versus merely “faster process”—this is my inference; the study itself made no such attribution. Third, this was a single-center pragmatic trial conducted at an academic medical center, unblinded—clinicians knew whether they had received AI-generated invitations; the trial was carried out by the same team that developed the model; and the third-party registry noted that neither the model nor the underlying data are accessible outside the institution. Fourth, a third-party registry verification found: no registration record for this trial exists on ClinicalTrials.gov, and the paper does not provide a registration number.
Notably, the study itself states its conclusions transparently. The abstract’s concluding sentence uses the word “**associated with**,” not “caused” or “led to”; the non-significant P = 0.07 appears verbatim in the abstract—not buried deep in the text; the non-significant secondary endpoint, post-randomization exclusions, and single-site design are all explicitly stated in the abstract, not concealed in some page of the main text.
## Then What?
Assembling this body of evidence, my assessment is as follows: First, conducting **randomized deployment evaluations** of already-deployed AI prediction models remains uncommon in the AI field—third-party registry searches by topic yielded only 16 related records, and aside from the team’s 2024 retrospective development paper, none described randomized evaluations. This trial raised the evidentiary bar: from “retrospective AUC calculation” to “deployment into workflow and impact on real-world outcomes.” Such trials are expensive, slow, and require hospitals willing to subject real clinical workflows to randomization—likely why they remain rare (my speculation). Second, the arc traced by this same team—from that AUC value to this two-year randomized pragmatic trial—is itself the story: bridging the gap between “the model can distinguish high-risk individuals” and “deploying the model gets patients diagnosed earlier” is precisely the chasm most AI tools never cross. The clinical AI field is not short of papers—it is short of such trials. Third, 43 days versus 138 days delivers tangible relief to patients enduring prolonged diagnostic odysseys before confirmation; but the next essential question concerns outcomes: do lupus patients diagnosed earlier experience better long-term kidney and joint outcomes? That requires longer follow-up; and to generalize beyond this site, multicenter trials validating external validity are indispensable—this step cannot be bypassed. Fourth, including the non-significant secondary endpoint in the abstract, openly reporting post-randomization exclusions and analytic sample sizes—this level of transparency regarding limitations is more worthy of industry emulation than any AUC claim. It leaves readers with critical questions: When deciding whether to reimburse or purchase an AI triage tool, should payers and hospitals set “randomized deployment evidence” as a threshold? And in settings where specialist appointments are scarce and hard to obtain, might AI triage soon become standard—then who ensures fairness in its patient selection?