AI citations linking to a real source that doesn't support the claim
Geeky

AI Citations: The Link Is Real, the Claim Isn’t

I was reading up on Ozempic side effects when an article made a claim I wasn’t sure about. So I did what a lot of people do now: I asked Gemini to check it. Gemini said the claim was correct and pointed me to a study.

I opened the study. It had nothing to do with the claim.

The link worked, the study was real, and the topic was close enough to look right at a glance. It just didn’t say what Gemini said it said. Of all the problems with AI citations, that’s the one that worries me most, because it survives a quick look.

Fake links are the easy problem with AI citations

When people talk about AI and sources, they usually mean invented references: a paper that doesn’t exist, or a URL that leads to an error page. I covered that side in my post on AI in journalism. A fake link is annoying, but it gives itself away the moment you click.

A real link that doesn’t support the claim is harder to catch. The page loads. It’s from a credible site. It’s about the right subject. Unless you read it looking for the specific sentence, you’ll walk away thinking the claim was verified.

And it isn’t a one-off. It has happened to me many times, with almost every AI agent I’ve tried. Gemini is where it bit me hardest, but I pay for Claude, ChatGPT, and Google’s AI plan, and I don’t treat any of them as exempt.

What the research says

Studies measuring how often AI citations fail to support their claims

The best evidence for this exact failure comes from medicine. A Stanford-led team built an automated checker called SourceCheckup and ran seven popular models through 800 medical questions. That produced about 58,000 pairs of statements and the sources cited for them.

The results, published in Nature Communications in April 2025, are rough. Between 50% and 90% of responses weren’t fully supported by their own sources, and some were contradicted by them. Even GPT-4o with web search left about 30% of individual statements unsupported. Doctors reviewed the findings independently and backed them up.

News isn’t any better. Columbia’s Tow Center tested eight AI search tools on 1,600 queries, asking each one to identify the source of an excerpt from a real article. Together, they got more than 60% wrong.

Then there’s the News Integrity in AI Assistants study from the European Broadcasting Union and the BBC: 22 public broadcasters, 18 countries, and more than 3,000 answers. 45% had at least one significant issue, and 31% had serious sourcing problems, including blaming a real news outlet for information it never published.

StudyWhat it testedHeadline finding
SourceCheckup (Nature Communications, April 2025)Medical answers against the sources they cited50–90% of responses not fully supported
Tow Center (March 2025)Finding the source of news excerptsWrong on more than 60% of queries
EBU and BBC (October 2025)News answers from four assistants31% had serious sourcing problems

One caveat. All three studies are from 2025, and the models have moved on since. I couldn’t find newer editions of any of them. So read these as proof that the problem is real and common, not as today’s exact error rates. From what I keep running into, I’m not convinced it has gone away.

Naming names

Gemini, the tool that burned me on the Ozempic claim, had the worst sourcing record in the EBU and BBC study. 72% of its answers had a significant sourcing issue. The other three assistants, ChatGPT, Copilot, and Perplexity, were all below 25%.

In the Tow Center test, Grok 3 got 94% of queries wrong, and 154 of its 200 citations led to error pages. ChatGPT misidentified 134 of 200 articles and never once declined to answer. Perplexity did best of the bunch and still got 37% wrong.

Paying more didn’t help either. Perplexity Pro and Grok 3 got more answers right than their free versions, but they also got more wrong, because they answered confidently instead of admitting they didn’t know.

Claude wasn’t in either news study, so I can’t give you a number for it. My own experience says it isn’t immune.

Why a real link can back a wrong claim

Here’s the simple version of what’s going on. When an AI tool searches the web, it pulls in pages about your topic. Then it writes an answer, partly from those pages and partly from what it learned in training. The citations get matched to sentences along the way, and the tool tends to pick whichever page looks most related.

“Related” and “supports this exact sentence” are not the same thing. A study about a drug’s side effects in general sits very close to a claim about one specific side effect. To a model matching topics, that’s a fit. To a reader, it’s a citation that proves nothing.

That’s my reading of how these tools behave, not something any vendor has spelled out. But it fits the pattern. The bad citations are rarely random. They’re almost right, and that’s exactly what makes them convincing.

It also explains why my Ozempic check failed. I asked the AI to verify a claim, and it did what it always does: it found something nearby and said yes. Asking an AI to fact-check a claim gets you a second answer, not a check.

How I check AI citations now

Checking AI citations by finding the exact claim on the source page

My rule is simple, and it’s strict on purpose:

  1. Open at least one link per claim. Pick the source you trust most, not just the first one in the list.
  2. Look for the claim, not the topic. The page has to say what the AI said, not just discuss the same subject.
  3. If it isn’t there, drop the whole claim. I don’t trim it or soften it. If the main source doesn’t back it, I treat it as misinformation.
  4. If the main source does say it, accept it. You don’t need five confirmations. You need one real one.

Step 3 is the one people skip. It feels wasteful to throw out a claim that might be true. But an answer with a citation that doesn’t hold up has given you no evidence at all, and “might be true” is exactly where misinformation lives.

Pro Tip: On a long page, press Ctrl+F (Cmd+F on a Mac) and search for a key number or phrase from the AI’s answer. If it isn’t on the page, the citation doesn’t support it.

Treat the link as a lead

AI tools are good at finding pages I wouldn’t have found on my own, and that’s the right job for them: pointing you at sources. The mistake is letting the little citation number do the reading for you.

An AI citation is a lead. It becomes evidence only after you’ve opened it and found the sentence. Until then, it’s just a link that looks like proof.

Keep reading about: AIEducationGoogleInternetScience

Related

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.