Hallucinations: causes and concrete mitigations

A confidently wrong AI answer can lose a customer or a court case. This lesson is the practical defense.

A friend who never says "I do not know". Charming, dangerous, eventually fired from any serious job.

Hallucinations happen because LLMs sample probable tokens, not retrieve facts. Causes:

• Asked about post-training events
• Asked about your private data the model never saw
• Long context with the answer in the middle (lost)
• Vague prompts that leave too much to the model

Mitigations:

1. RAG with strict refusal rules
2. Tool use: search, calculators, DB lookups for facts
3. Citations: require sources
4. Temperature 0 for factual tasks
5. Verifier model: a second LLM that fact-checks against sources
6. Human in the loop for high-stakes use cases

A verifier pattern:

def answer_then_verify(question):
    ans = generate_with_rag(question)
    verdict = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "system", "content": "Reply YES if the answer is fully grounded in the sources, otherwise NO with reason."},
                  {"role": "user", "content": f"Sources:\n{ans['sources']}\n\nAnswer:\n{ans['text']}"}]
    )
    return ans, verdict.choices[0].message.content