In November 2025, something unusual happened during the review season of ICLR—the top-tier machine learning conference. Researcher Micah Goldblum exposed on social media a submission entirely generated by a large language model (LLM), stuffed with jargon and fabricated references—yet it ranked in the top 17% of all submissions, receiving two scores of 8. Fortunately, one reviewer who actually read the paper gave it a 0. Another researcher added that the same paper had been submitted at least four times under different titles. ICLR’s official account later confirmed the paper had been outright rejected and notified reviewers handling its other versions. On November 19—just days later—ICLR’s Program Chair issued a statement announcing two measures: using LLM-detection tools to screen both submissions and reviews; and an unusually symmetrical penalty—reviewers who write low-quality AI-generated reviews may have their own papers summarily rejected.
This incident warrants pause for everyone who cares about science: the problem isn’t isolated to a single step—it’s the entire trust infrastructure underpinning science, now simultaneously breached at both ends by AI.
## Production End: Writing a Paper Is Nearly Human-Free
First, consider the production end—the side where the incident occurred. This is not a new problem emerging in 2025. In 2023, the number of retracted research papers worldwide surpassed 10,000 for the first time, shattering the annual record—Nature described it as "shattering the annual record," citing publishers’ exhaustion from cleaning up fake papers and peer-review fraud. Hindawi alone retracted over 8,000 papers; according to Retraction Watch, this exceeded the total number retracted by all publishers combined in any prior year. That year, four of Hindawi’s journals were shut down after being overwhelmed by paper mills, and nearly 20 journals were delisted from major indexes. Paper mills—a gray-market industry specializing in mass-produced, ghostwritten, and review-manipulated submissions—had already compromised peer review before LLMs arrived; with LLMs, the marginal cost of producing a 'plausible' paper has been further driven toward zero.
A more direct signal comes from AI company Sakana. In March 2025, it announced that its AI Scientist-v2 system had end-to-end generated a paper—from research hypothesis, experimental code, running experiments, data analysis, to full-text writing—that passed double-blind review for an ICLR 2025 workshop, scoring slightly above the acceptance threshold. Sakana called it "the first fully AI-generated paper, to our knowledge, to pass the same review process as human scientists."
Let’s be clear about the caveats: this was a workshop, not the main conference (workshop acceptance rates are typically far higher); the experiment itself was pre-coordinated with the workshop organizers, and acceptance came with immediate withdrawal; Sakana itself admitted none of its three submissions met main-conference standards; and Sakana’s internal audit found the system made citation errors—e.g., attributing LSTM to a 2016 reference, while the original LSTM paper was published in 1997. Yet it remains a milestone: the review process failed to detect this as machine-written. The production end has been breached.
## Review End: The Immune System Itself Is Infected
The truly troubling part lies at the other end. Science’s immune system—peer review—is built upon an unspoken assumption: reviewers’ attention is scarce and human. Today, that attention is increasingly outsourced to AI.
During the ICLR crisis, AI text detection firm Pangram analyzed all submissions and reviews for that season (vendor-reported figures, to be taken with caution): 21% of reviews showed signs of AI generation, and over half involved some degree of AI assistance; papers reviewed by high-AI-assistance reviewers scored lower, whereas reviews written by AI tended to assign higher scores and run longer. While ICLR’s Program Chair did not disclose full-scale proportions in the post-mortem, he acknowledged the scale: of 76,139 reviews this cycle, "human judgment alone cannot consistently identify problematic cases." They instead ran two detectors and flagged reviews deemed fully AI-generated by both for domain chairs.
Why outsource review? Structural reasons outweigh moral ones: reviewing is unpaid and overwhelming—one top-tier conference mobilizes over ten thousand reviewers (ICLR 2026 used 18,054). As the editor-in-chief of Science stated back in 2023, most academic misconduct handled by journals stems from "insufficient human attention." When outsourcing tools appear, outsourcing becomes an economic imperative—not individual corruption.
## Institutions Are Patching Holes—But Always Playing Catch-Up
Journals and conferences reacted swiftly—and uniformly. In January 2023, Nature declared LLMs ineligible for authorship, stating plainly: authorship implies responsibility, and AI cannot bear responsibility; Science went further, banning all AI-generated text outright—violations are treated as seriously as image manipulation or plagiarism. Science’s current policy also forbids reviewers from inputting any part of a manuscript into an LLM—this violates confidentiality obligations. ICLR’s policy explicitly states: even if an entire study is conducted by an LLM, a human author must assume responsibility.
The most interesting implementation is ICLR’s hallucinated-citation pipeline: automatically extracting each submission’s references and cross-checking them against multiple bibliographic databases and web searches. Machine false positives are nontrivial (e.g., mistaking translated non-English titles for fake citations), so every flagged citation undergoes manual verification—first by domain chairs, then by the Program Chair reviewing each case individually. At least three humans examine each flagged paper; confirmed fabrications trigger immediate rejection, with an appeals channel available. The official post-mortem admits this pipeline "partially explains this year’s elevated outright rejection rate": of 19,525 valid submissions, 779 were rejected outright, and over 5,000 were withdrawn mid-process, yielding a final acceptance rate of 27.4%.
Yet all these are stopgaps. In 2023, Nature asked and answered its own question: Can editors and publishers reliably detect LLM text? "At present, the answer is 'maybe.'" And "LLMs will improve—quickly." ICLR’s post-mortem similarly concedes detector accuracy is "far from perfect"; they dare not penalize reviewers who merely use AI for light editing or whose judgments remain sound. Disclosure relies on self-reporting, detection on an arms race, and punishment on post-hoc investigation—three lines of defense, all chasing, none leading.
## My Three Judgments
First, papers are signals of scientific trust—not scientific products; the product is reliable knowledge. A signal is credible only because it carries two guarantees: production is costly (running experiments and writing papers takes years), and review is scarce (peers carefully read the work). AI nullifies both guarantees—text can be infinitely mass-produced, and review can be outsourced. Signal devaluation is not a moral failure but an economic inevitability. The record-breaking retraction count is merely a lagging indicator on the books.
Second, the review crisis is less a moral crisis than a pricing crisis. Peer review has long operated as a gift economy: reviewing a paper takes hours, pays nothing, and relies on norms of reciprocity within the scholarly community. With LLMs, this "gift" now has a zero-cost substitute for the first time. Relying on voluntary adherence to norms is equivalent to having no defense at all; ICLR’s symmetrical penalty—your paper gets rejected if you submit a shoddy AI-written review—essentially sets a steep, one-time price for breach. This is more effective than any appeal—and explains why its announcement became news.
Third, the root solution lies not in hunting AI text, but in shifting trust’s anchor from text to artifacts. In an era where text costs approach zero, trust must anchor in things whose cost cannot be zeroed out: executable code, raw data, reproducible workflows, pre-registered hypotheses, and citations verifiable line-by-line. ICLR’s hallucinated-citation pipeline is a prototype of artifact verification—citations are among the few elements in a paper that machines can hard-match against databases. But let’s be clear: not all disciplines possess executable artifacts (pure mathematics relies solely on proofs), and artifact review still consumes scarce human attention. This is a direction—not a panacea.
## Four Takeaways
First, if you do research: disclosing AI use does not devalue your work—failing to disclose does. Disclosure is now codified in ethics guidelines. Do not outsource review: confidentiality and accountability are hard constraints, and the cost of breach now has a real price.
Second, if you read papers: a paper is not truth. Check whether it has been retracted; verify whether data and code are publicly available; watch for pre-registration; treat unusually smooth yet hollow text with extra skepticism—Nature warned long ago that LLM text tends to be "bland and generic."
Third, if you’re merely a scientific bystander: science’s self-correction has never been automatic—it is paid for by the attention of thousands of anonymous reviewers. Record-high retractions are not science’s scandal, but evidence that the correction mechanism is still operating—albeit under unprecedented strain.
Fourth, for everyone: this is not just science’s problem. Any institution whose trust rests on text—papers, news, resumes, financial reports, court judgments—faces the same structural shock: when "writing a credible passage" no longer requires a human, credibility must come from outside the text. Science is simply the first industry forced to answer.
In ICLR’s story this season, the sharpest detail is that 0 score: a paper that fooled the average score was stopped by one reviewer who actually read it. The future of the system may well lie in those reviewers willing to spend those few hours. Have you encountered AI-text that feels "smooth but hollow"? Share in the comments.
Source: https://blog.iclr.cc/2026-03-31/a-retrospective-on-the-iclr-2026-review-process/