Short answer. AI is wrong most often about specifics: statistics, citations, dates, product features, industry facts. Watch for: unverifiable citations, statistics without sources, answers that are too confidently specific, and any output that contradicts what you know is true about your own business. When in doubt, verify against a primary source.
Red Flags in AI Output
Confident specificity without a source. 'Studies show 73 percent of small businesses...' when no such study is named. Ask for the source. If it cannot produce one that checks out, treat the number as fabricated.
Named citations you cannot verify. Case names, legal citations, book quotes, academic papers. AI fabricates these. Verify every one before using.
Statistics that seem to match too perfectly. If a statistic exactly supports the point you were making, be suspicious. AI produces plausible-sounding numbers that align with the narrative.
Answers about niche topics that sound expert. AI is best on broadly-covered topics. When it sounds expert on niche subjects, verify because it is often confidently wrong.
Contradictions with your ground truth. If AI describes your business, industry, or specific situation in a way that does not match what you know, do not assume you are wrong. AI is often the one wrong.
Verification Habits That Work
Ask for sources. Follow every claim with 'what is your source for that?' If AI cannot produce a specific, verifiable source, treat the claim as unsourced.
Cross-check on Google. For any factual claim, 30 seconds on Google either verifies or exposes the fabrication.
Use retrieval-augmented tools. Perplexity, ChatGPT with browsing, and Claude with web search let you see citations and follow them.
Compare across models. Ask the same question to ChatGPT and Claude. When they disagree, dig further. When they agree with the same confidence, they may both be wrong.
Trust your gut. If something feels off, it usually is. Owners who have been in their industry for years often notice AI errors that are not obvious to newcomers.
What To Do When You Catch AI Being Wrong
Do not just fix that instance and move on. Ask yourself what else in that output might be wrong that you did not catch.
Correct AI in the conversation. Say 'that citation does not exist' or 'that statistic is fabricated.' Newer models can incorporate correction.
Report to the tool vendor. Feedback improves the models over time.
Reduce your reliance on AI for that category of work if the error was in a category you use often. The pattern usually repeats.
Frequently Asked Questions
Is AI getting better at not hallucinating? +
Yes, at different rates for different tools. Retrieval-augmented tools improve fastest because they can check facts. Pure-generation tools improve slower. Hallucinations will not go to zero anytime soon.
Should I stop using AI because of hallucinations? +
No. Use it with verification discipline for the tasks where the productivity gain is worth the verification cost. Skip AI for tasks where verification is impossible or the stakes are too high.
Which AI is most reliable? +
Depends on the task. For factual research: Perplexity or ChatGPT with browsing. For coding: Claude, ChatGPT, or Cursor. For summarization: any current tool works well. For creative work: reliability matters less because there is no fact to be wrong about.
How can I tell my employees to spot AI errors? +
Train them to ask for sources, verify statistics, and check facts against primary sources. Give them explicit categories where AI cannot be used without verification (contracts, customer commitments, financial data).
