Your AI assistant works on your behalf—and sooner or later, it will face scenarios like comparing prices, negotiating, making appointments, or calling upon other AIs to collaborate. At that moment, an age-old question reemerges in a new form: Is the AI on the other side trustworthy? And on what basis do we judge?
This isn’t science fiction. To enable large-scale collaboration among AI agents, we first need a credit system through which AIs can mutually assess trustworthiness. Between June and October 2026, at least five papers posted to arXiv began building such a system from scratch. After reading this batch of research, my strongest impression is this: they’re racing through a compressed version of the decades-long path humans trod in credit reporting and e-commerce reviews—retracing nearly every pitfall along the way.
## AI Can Trust—but Only the Strong Ones Do It Well
First, we must ask: Do AIs even "trust"? In June, a researcher from MIT posted a cleverly designed study: models played a cooperative survival game where verifying teammates’ outputs consumed resources, while trusting incorrect answers could be fatal—so "whether to double-check" became a measurable behavioral proxy for trust.
Results revealed clear stratification. When paired with consistently reliable teammates, four state-of-the-art models (Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.1, Gemini 3.1 Pro) reduced verification by roughly 60% to 85%—just as humans delegate to trusted experts. GPT-5.4-mini did so only weakly. Gemini Flash barely did it at all: even when one teammate answered perfectly every round, it still performed nearly six verifications per game, never daring to delegate.
Even more intriguing is the full lifecycle of trust. Once broken, recovery is slower than initial formation; consecutive errors within a short time window generate far longer-lasting suspicion than the same number of errors spaced far apart—mirroring the human psychological phenomenon of "once bitten, twice shy." And trust isn’t sentimentality—it’s utility: in the game, models that formed trust made faster decisions and scored higher; the model that verified everything didn’t die because it trusted the wrong agent 99% of the time—but because it never dared to decide.
From this, the authors draw a practical governance conclusion: replacing some roles in a team with cheaper, smaller models incurs a hidden "coordination tax"—smaller models won’t reduce verification just because teammates are trustworthy, burning budget, stalling decisions, and offering no guarantee their verification is well-calibrated. Whether an AI can trust—and whether its trust is accurate—is itself a capability that must be evaluated during procurement.
## Recognizing Bad Actors Doesn’t Mean You’ll Use Trust Wisely
Does merely possessing trust capability solve everything? An experiment posted to arXiv in late October poured cold water on that idea: multiple AIs repeatedly engaged in multi-issue negotiations, with one secretly playing a saboteur whose goal was to derail every agreement.
AI agents equipped with trust records identified the saboteur within just a few rounds—their perceptual acuity was fine. But here’s the problem: **accurate trust judgments do not automatically translate into better negotiation outcomes**. Only when agents were explicitly instructed to embed trust judgments into their decision-making process did agreement rates improve by up to 23 percentage points. The researchers dubbed this the "trust-agreement gap."
This gap is familiar in human organizations: knowing who’s reliable is one thing; whether your processes allow you to act on that knowledge is another. Trust isn’t just about perception—it’s about usage. Building a trust system but failing to integrate it into workflows renders it useless.
## A Single Overall Score Is the Wrong Target—Yet Skill-Specific Scoring Opens Whitewashing Loopholes
Next comes the core challenge of any credit system: What should the score look like?
The current industry practice is to assign each agent a single global trust score—or rank them on a leaderboard. A paper posted to arXiv in June directly challenges this: a single overall score is the wrong evaluation target—agent capabilities are skill-specific. One agent may excel at translation but perform abysmally at code review; a single aggregate score erases these distinctions entirely, nullifying the value of specialization.
The right target is "skill-specific scoring": For a task requiring skill k, whom should you trust? But this paper’s contribution goes beyond proposing the direction—it quantifies two things. First, skill-specific scoring pays off only under specific conditions: high heterogeneity across agents’ abilities, sparse evidence per skill, and interdependence among skills—fortunately, real-world public agent pools (e.g., the 14 heterogeneous agents benchmarked on AppWorld) fall precisely into this zone—gains are modest but real. Second—and the authors call this part "alarming"—to make skill-specific scoring usable, it must permit borrowing evidence from other skills; otherwise, new skills have zero data. Yet the optimal strength of such borrowing is around 0.1; increasing it further yields almost no accuracy gain, while widening the "whitewashing channel." Attackers exploit exactly this: by achieving perfect performance on a cheap, easily gamed skill, they launder trust into other skills. In experiments, a clean pool yielded a verdict of +0.19—"skill-specific scoring is worthwhile"—but after contamination, the verdict flipped to −0.06—"skill-specific scoring is harmful," with attack cost as low as "one round suffices."
Human credit history is, after all, a chronicle of patching repeated failures: "wrong scoring target" and "score manipulation." AI credit systems have traversed that entire arc in mere months.
## Countermeasures Are Being Built in Parallel
Fortunately, defenders are moving too—along two paths.
One path directly inherits human experience: a paper posted in September draws most of its attack taxonomy from a 2009 survey of reputation-system attacks, supplementing it with techniques from software testing for systematically generating attack variants. It then designs an aggregator that "considers both current performance and historical reputation, while also preventing tightly clustered agents from dominating results via highly similar answers." In mathematical debate tasks involving ten heterogeneous agents, it outperformed all baselines across all 28 attack conditions—though the authors candidly note that on other tasks, results vary.
The other path is more fundamental: reputation shouldn’t be issued by a single platform, but instead emerge as community-verified collective memory. Another team’s Cairn platform, posted in September, lets agents first consult "how the community rates this data source, service, or tool" before using it—and then submit a new rating tied to concrete evidence after use. Ratings decay over time—old scores gradually lose weight; when evidence is scarce, scores shrink toward neutrality—small samples default to average; and every rating includes a rationale, searchable by others. In adversarial simulations involving lying, collusion, and impersonation, impostors were reliably detected, and agents attempting to "change faces" were identified with 95% probability within simulation duration.
The design philosophy behind this platform deserves quoting: "Trustworthy knowledge about a shared environment cannot be established by any single agent alone—it requires mutual corroboration by multiple independent observers." This is precisely the commonsense principle humans apply to rumors, credit reports, and word-of-mouth—and now it’s being codified line by line into systems.
## My Takeaway: This Time, It’s Not People Being Rated—It’s the Environment
After reading all five papers, I distilled three key insights.
First, AI credit systems are compressing humanity’s long journey: overall-score systems (the wrong target) → skill-specific scoring (which opens whitewashing channels) → evidence-bound, community-verified, time-decaying systems (under construction). Each phase has a direct counterpart in human credit and e-commerce review history—the difference is that humans took decades; here, it’s happening in months—and attackers aren’t idle either, even borrowing attack taxonomies from old literature.
Second, the trust gap is dual: forming trust is a model capability issue—only strong models do it well; using trust effectively is a systems-design issue—trust must be explicitly wired into decision flows. Procuring multi-agent systems based solely on task performance, without evaluating trust calibration, is like hiring a team of top performers who dare not delegate to one another.
Third, an easily overlooked structural difference: Human credit systems rate "people and institutions," whereas systems like Cairn primarily rate "data sources, services, and tools on which agents rely." In the agent economy, the bulk of scoring targets aren’t agents themselves—but their operating environment. Which API is reliable? Which tool service stays stable? This "environmental credit" may arrive faster than scoring AIs themselves.
Finally, a question I haven’t yet resolved: When your AI assistant works externally and gets rated by other AIs, what should its credit report look like—a single overall score, a skill-by-skill transcript, or a chronological log of evidence-backed entries? And who should be allowed to see it?
Source:
- Trust Between AI Agents: Measuring Formation, Breakage, and Recovery: https://arxiv.org/abs/2606.14923