Never trust one AI's quote: spot AI hallucinations by checking the source
This post will help you check information from an AI tool. You will learn how to compare AI-generated text with original documents. This will help you get accurate facts and figures.
Why checking the source matters
AI tools can create text that sounds convincing, but it might not always be correct. Sometimes, an AI makes up facts or figures, or it misquotes a document. This is sometimes called an 'AI hallucination'. It is important to spot AI hallucinations by checking the source document yourself. This is especially true when you need precise information, such as dates, numbers, or direct quotes. Relying on unverified AI output can lead to mistakes in your work or when you share information with others. Always go back to the original document to confirm what the AI tells you.
What to prepare before you start
Before you ask an AI tool for information, make sure you have the original document ready. This might be a PDF, a webpage, or a text file. You need to be able to open and search this document easily. If the document is very long, it helps to know which sections are most likely to contain the information you are looking for. Having the source document open and ready to search will save you time when you need to verify the AI's answer. Make sure you are using a reliable version of the document, not an older draft or an unofficial copy.
How to ask the AI for specific information
When you ask an AI tool for information from a document, be clear and specific. Tell the AI to refer only to the document you provide or link to. For example, you might ask what a named section of the document says about one topic, and ask for the exact words. The more precise your question, the better chance the AI has of giving you a useful starting point. Avoid asking open-ended questions that invite the AI to summarise or interpret too much. Focus on asking for facts, figures, and direct quotes that you can easily check against the source.
Step-by-step: checking the AI's answer
Once the AI gives you an answer, follow these steps to verify it:
- Locate the relevant section: Go to the original document and find the part that the AI's answer refers to. If the AI does not tell you where it found the information, you will need to search the document for keywords from the AI's response.
- Compare facts and figures: Check every number, date, and name the AI provided against the document. Make sure they match exactly.
- Verify quotes: If the AI gave you a quote, compare it word-for-word with the original text. Even small changes can alter the meaning.
- Check context: Read the surrounding sentences in the document. Sometimes, a quote or fact can be accurate but used out of context by the AI, which changes its meaning.
This careful comparison helps you spot AI hallucinations by checking the source directly.
What to do if the AI's answer is wrong
If you find that the AI's answer does not match the source document, do not use the AI's information. Instead, use the information directly from the original document. You can also tell the AI that its answer was incorrect and provide the correct information from the source. For example, you might say: 'The document actually says this, not that.' If the AI and the source disagree, use what the source document says.
Common problems and how to fix them
One common problem is when an AI provides a quote that is slightly different from the original. It might change a few words, add or remove punctuation, or combine sentences. To fix this, always copy the quote directly from the source document yourself. Another issue is when an AI gives you a figure that is close but not exact. Double-check every digit. If the document is long or complex, the AI might struggle to find specific details. In these cases, try to guide the AI by telling it which section or page to look at, or ask it to search for very specific phrases. For more tips on setting clear rules for your AI, see our post on custom instructions.
Keeping control over AI-generated information
Regularly verifying AI outputs helps you develop a critical approach to using these tools. It is a good habit to always question and confirm information, especially when it is important. This practice builds your confidence in using AI tools effectively. It also helps you catch mistakes before you share them. By taking control of the verification process, you ensure that the information you use is accurate and reliable. Remember to ask for a brief, not a chat, to get more focused responses from your AI. You can learn more about this in our post: Ask for a brief, not a chat.
Next steps for your own work
Practise these verification steps with different documents and AI tools. The more you check, the better you will become at quickly finding and confirming facts. Make it a routine part of your workflow when using AI for research or content creation. This will help you rely on AI tools more confidently for your tasks.
If you would like support to build this skill, AI mentoring is aXai's program for people with disability who want to learn AI tools for their own goals, at their own pace. You can register your interest on the page.
aXai is a disability-led AI, software and accessibility company, delivering through AI-supported work: skilled people with disability in real, properly paid roles, with AI removing the barriers. Read how AI-supported work works.
Easy Read version
Check AI information
AI tools can give you information.
Sometimes the information is wrong.
This is called an AI hallucination.
This post helps you check AI information.
You will learn how to compare AI text with original documents.
This helps you get correct facts.
Why checking is important
AI text can sound right.
But it might not be correct.
An AI can make up facts.
It can misquote a document.
You need to check the original document yourself.
This is important for exact information.
This includes dates, numbers, or direct quotes.
If you use wrong AI information, you can make mistakes.
Always check the original document.
What to get ready
Before you ask an AI tool, get the original document.
This could be a PDF or a webpage.
You need to be able to open and search it.
If the document is long, know which parts have the information.
Having the document open saves time.
Use a correct version of the document.
How to ask the AI
When you ask the AI, be clear.
Tell the AI to use only the document you give it.
For example, ask what the document says about one topic.
Or ask for the exact words from the document about a detail.
Clear questions help the AI give a useful answer.
Do not ask open questions.
Ask for facts, figures, and quotes.
You can easily check these against the source.
How to check the AI's answer
When the AI gives an answer, check it.
-
Find the right part: Go to the original document. Find the part the AI talks about. If the AI does not say where it found the information, search the document. Use words from the AI's answer.
-
Compare facts: Check every number, date, and name. Make sure they are exactly the same.
-
Check quotes: If the AI gave a quote, compare it word-for-word. Small changes can change the meaning.
-
Check context: Read the sentences around the information in the document. A fact can be right but used in the wrong way by the AI. This changes its meaning.
This checking helps you find AI mistakes.
What if the AI is wrong?
If the AI's answer is wrong, do not use it.
Use the information from the original document.
You can tell the AI its answer was wrong.
Give the AI the correct information from the document.
For example, say: 'The document actually says this, not that.'
If the AI and the source disagree, use what the source document says.
Common problems and how to fix them
Sometimes an AI quote is a bit different from the original.
It might change words or punctuation.
Always copy the quote directly from the document yourself.
Sometimes an AI gives a number that is close but not exact.
Check every digit.
If the document is long, the AI might miss details.
Tell the AI which section or page to look at.
Or ask it to search for exact phrases.
For more tips, see our post about custom instructions.
Keep control over AI information
Checking AI answers helps you use AI tools well.
Always question and confirm information.
This is a good habit.
This helps you catch mistakes before you share them.
By checking, you make sure your information is correct.
Ask for a brief, not a chat, to get better answers from your AI.
Learn more in our post: Ask for a brief, not a chat.
Next steps for your work
Practise checking with different documents and AI tools.
The more you check, the better you will get.
Make this a regular part of your work.
This helps you use AI tools with more confidence.
AI mentoring is aXai's program for people with disability.
You learn to use AI tools for your own goals, at your own pace.
You can register your interest on the page.
aXai is a company led by people with disability.
Skilled people with disability do the work.
They have real jobs with proper pay.
AI removes the barriers.