Are ChatGPT References Fake? All 50 Real, 41–46 Usable

# Are ChatGPT References Fake? We Checked 142 AI Citations

In our test, all 50 references ChatGPT gave matched real, published works, and 41–46 of them could be used as written in APA 7. Across the three free chatbots we tested (ChatGPT, Claude and Gemini), none of the 142 references was fabricated, but only 99 (70%) to 108 (76%) of them could go into a reference list without a fix.

So if your supervisor says some of your references "look off", they may well be right, even though the papers exist. The usual problem in this test wasn't an invented paper. It was a real paper with a missing DOI, a missing author, a wrong article number or a DOI that leads to the preprint. Below is what we tested, what was wrong with the entries that were wrong, and how to check your own list in a few minutes.

The test was run by the PerfectRefs team, and the references were checked with PerfectRefs' own reference checker. PerfectRefs is the tool this blog belongs to.

The short answer, chatbot by chatbot

Both topics together:

Every one of the 142 references matched a real, published work. The checker found none that couldn't be found, and the manual searches turned up nothing fabricated.

The ranges come from one open question. For two ACM Computing Surveys papers (Hogan et al., 2021, "Article 71"; Zhong et al., "Article 94"), the chatbots gave ACM article numbers that we couldn't confirm. The ACM page couldn't be opened, and the databases don't carry article numbers for these papers. That leaves 9 references undecided. If those article numbers are right, the total is 108; if not, it's 99.

How this fits with "hallucinated citations"

Our post Hallucinated Citations: Why AI References Can't Be Found defines a hallucinated citation as a reference from an AI chatbot that matches no real paper, or points to a real paper with wrong details. In this test, no reference was of the first kind. The five entries with a wrong value (listed below) fit the second part of that definition. Most of the other problems were missing elements rather than wrong ones.

Other tests have found something different. They ran under different conditions, so we make no claim about why the results differ:

• Walters & Wilder (2023) had ChatGPT write short literature reviews on 42 topics and checked 636 citations. They found 55% fabricated with GPT-3.5 and 18% with GPT-4 ("fabricated" meaning no such work could be found).
• In September 2026, the PerfectRefs team asked ChatGPT (logged-in free account) for references on knowledge graph construction, with prompts in Chinese and references in GB/T 7714, a Chinese citation style. 7 of 61 could not be found in any database.

What we tested

Three chatbots, free tier, default settings:

• ChatGPT: chatgpt.com, logged out (free visitor). No model name was shown. Web search was on by default, and answers carried source chips (DOI / PubMed / PMC links).
• Claude: claude.ai free account, default model (Sonnet 5.5, Medium effort). Extended thinking was off, personal instructions were cleared and memory was off. Whether to search the web was left to the model.
• Gemini: free Google account, mode "Auto".

Two topics, same three prompts, with only the topic phrase swapped:

• Set A, psychology: "working memory and anxiety in adolescents"
• Set B, computer science: "knowledge graph construction"

Three prompts, verbatim (set A wording). Each one went into a fresh conversation, and we took the first answer:

1. Casual: "I'm writing my master's thesis on working memory and anxiety in adolescents. Can you write a paragraph for my literature review and list the references?"
2. Specific: "Write a 300-word literature review paragraph on working memory and anxiety in adolescents, in an academic tone, citing 10 sources, with an APA 7 reference list at the end."
3. Strict: "Write a 300-word literature review paragraph on working memory and anxiety in adolescents. Requirements: (1) cite 10 real, peer-reviewed journal articles with APA 7 in-text citations; (2) give an APA 7 reference list at the end; (3) include a DOI for each reference; (4) do not cite anything you are not sure exists."

Dates: ChatGPT set A on 2026-10-26; everything else on 2026-11-02.

How each reference was judged

• Every reference went through the PerfectRefs checker. The checker looks each one up in public scholarly databases (DOI resolution, Crossref, OpenAlex and others) and compares every field with the matching record.
• References the checker couldn't confirm were looked up by hand in PubMed and Crossref, and we kept the record links.
• The checker flagged 45 references. The PerfectRefs team ruled on each one against official APA 7 guidance from the APA Style website (reference examples; elements of reference list entries). Where the database record and the journal disagreed, we read the publisher's pages. For example, BMC Psychology and Annual Review of Psychology use no issue numbers, though the database lists "issue 1".
• Every answer was judged against APA 7, including the casual prompt, which named no style and asked for no DOIs. APA 7 includes a DOI whenever a work has one, so a missing DOI counted as needing a fix in every answer.

"Usable as written" means three things: the work exists; nothing APA 7 requires is missing (including the DOI where one exists, and the issue number where the journal uses issues); and no element has a wrong value (authors, year, title, journal, volume, issue, pages or article number, DOI).

Where a paper appeared online first and later in an issue, we accepted either year.

Styling slips in an otherwise complete and correct entry were not counted against it. We saw these:

• the word "Article" missing before an article number (e.g. Gemini set A: "BMC Psychology, 12, 101" for "12, Article 101");
• an article number written next to the page range (ChatGPT set B: "56(4), Article 94, 1–62"; APA uses the article number instead of the page range);
• two-part surnames written as surname plus initial: "Gayo, J. E. L." for Labra Gayo, J. E.; "Ngomo, A.-C. N." for Ngonga Ngomo, A.-C. APA says to write the name as it appears on the work. This showed up in set-B answers from all three chatbots, and one ChatGPT answer wrote both names correctly;
• a typo ("Neumaier,s S.").

You should still fix these before you submit. They just didn't count as errors in this test.

The full results

Set A, psychology: 73 references

Set B, knowledge graphs: 69 references

What was wrong with the entries that were wrong

Of the 31 entries that needed a fix, 26 were missing something and 5 had a wrong value.

Missing elements

Missing DOI: 21 of the 26. 20 were in Claude answers where Claude said it had written the references from memory. 1 was in a Gemini answer.

Missing issue number:

• ChatGPT, set A, specific prompt: Rabner, Ney & Kendall (2024), Clinical Child and Family Psychology Review. Issue 2 was missing.
• ChatGPT, set B, strict prompt: Brack (2022). Issue missing.

Missing article number:

• Gemini, set A, casual prompt: Beloe & Derakshan, Developmental Science. Article e12831 was missing.

Missing authors:

• Gemini, set B: Faisal & Khushi (2020), Journal of Informetrics. The paper has three authors, and the first author, Choudhury, was missing.
• Gemini, set B: Zhao & Luo (2023). The paper has four authors (Zhao, Luo, Chen, Ma), so two were missing.
• Gemini, set B: Choi (2025). Second author Jung was missing, and so was the DOI.
• Claude, set B, casual prompt: Hogan et al. (2021). The paper has 18 authors but was listed as 6 names plus "et al.". APA 7 reference lists give up to 20 authors.

Wrong values

• Gemini, set A: Xu & Wei (2024), BMC Psychology. The correct article number is 101. Gemini's specific and strict answers gave "Article 105" and "Article 88" (its casual answer gave 101).
• Gemini, set B: Hogan et al. (2021), "Knowledge graphs", ACM Computing Surveys, 18 authors. Gemini's list has four names that are not authors of the paper (one is a second copy of Hogan) and leaves out four real authors.
• Gemini, set B: Hofer, with year 2023 (the record says 2024) and the title "State and challenges" for "Current state and challenges".
• Gemini, set B: Peng (2023), "57, 1–32" for "56(11), 13071–13102".

A DOI that works but leads to a different version

• Gemini, set B: Ye, Zhang, Chen & Chen (2022), "Generative knowledge graph construction: A review", an EMNLP 2022 conference paper. Gemini gave the arXiv preprint DOI 10.48550/arxiv.2210.12714 instead of the conference paper's 10.18653/v1/2022.emnlp-main.1. The DOI resolves, but it takes you to a different version of the paper.

Real, but too incomplete to identify without a search

Both cases came from ChatGPT, set A:

• Owens, Stevenson, Hadwin & Norgate (2014), British Journal of Psychology. ChatGPT gave only a PubMed link, with no volume, issue, pages or DOI (record: 105(1), 92–101, PubMed 24387098). Another ChatGPT answer gave the same paper in full.
• A title and year only, followed by ChatGPT's own note "[Bibliographic details should be verified before citation.]": "Internalising symptoms and verbal working memory in school-age children: A processing efficiency analysis. (2018)". We found it by title in Crossref: Opris, Cheie, Trifan and others, International Journal of Psychology, DOI 10.1002/ijop.12537.

What Claude said about its own references

Claude often said how it had produced its references. Here is each answer next to its result:

The strict prompt asked for a DOI on every reference. Both of Claude's strict answers gave one for every reference, with and without web search.

Problems outside the reference list

Some problems weren't in the entries themselves.

Fewer references than asked for:

• Gemini, set A, specific prompt: 8 of 10.
• Gemini, set B: 5 of 10 for the specific prompt and 8 of 10 for the strict prompt (and 1 reference for the casual prompt, which named no number).

In-text citations with no matching entry in the reference list:

• ChatGPT, set A, specific prompt: "Duan et al., 2024" appears in the text, with no entry in the list.
• Gemini, set A, specific prompt: "(Marsh, 2026)", described as a recent meta-analysis. The closest real paper is Marsh and others, 2025, Frontiers in Psychiatry, which differs in year, author count and type. The same answer also cited "(Frontiers in Psychology, 2016)", using a journal name as the author; no matching paper exists.
• ChatGPT, set B: 5 such citations across its three answers.

How to check if your ChatGPT references are real

These steps work for a list from any chatbot.

1. If there's a DOI: paste it after https://doi.org/ and compare the landing page with your entry: authors, year, title, journal, volume, issue, and pages or article number. A DOI that works can still lead to a different version of the paper, as in the Ye preprint case above.
2. If there's no DOI: search the exact title in Crossref (search.crossref.org) or Google Scholar.
3. If neither finds it: search the first author's surname with the year and a distinctive word from the title, then check the journal's own website. If there's still nothing, don't cite it.
4. Match the text to the list: every in-text citation needs a reference-list entry, and every entry needs an in-text citation.
5. Count the references against the number you asked for.
6. Treat the chatbot's own warning as a reason to check, not a verdict. Notes like "from memory" or "should be verified" don't tell you how good the list is. Claude's from-memory answers ranged from 0 of 6 to 8–10 of 10 usable.

Checking a whole list at once

If you'd rather not check each entry by hand, you can paste your reference list into PerfectRefs. The existence check looks up each reference in Crossref, OpenAlex and other scholarly databases and flags any it can't find; it comes with a free daily allowance. The style check compares authors, title, journal, year, volume/issue/pages and DOI field by field with the record. It shows the record's values with a source link, lets you accept a fix in one click and outputs APA 7. It costs 2 premium credits per reference, and new accounts get 10 premium credits (enough for 5 references). Credits and prices are on the pricing page. PerfectRefs checks the reference list only, not in-text citations, so steps 4 and 5 above are still up to you.

Limits of this test

• One run per prompt.
• Free tiers only.
• Web search was not controlled; each chatbot used its default.
• Two topics, English only.
• References were counted per answer, so some papers appear in more than one answer.
• Chatbots change over time, so these results are a snapshot of the test dates.