Read the logprobs, not the answer
Every token a model emits comes with a probability distribution over the whole vocabulary. The decoded text only shows you the winner. The logprobs show you the race.
When you pull them back, scan for the spots where the top token barely beat the runner-up. Those are the seams. Names, dates, citations, and the first token of a refusal are where you most often find a coin-flip dressed up as a confident sentence.
A practical loop: request the top 5 logprobs per token, render them as a heatmap over the output, and read the cold spots first. That is usually where the hallucination lives.
Confidence ≠ correctness. But low confidence is a great place to look for hallucinations.
Aha moment
The final text is only the winning token path. The uncertainty is still visible in the runners-up.
Try this
Render low-margin tokens first, especially names, dates, citations, and first tokens after a policy boundary.
Watch for
- Fluent spans built from low-confidence token choices
- Nearly tied alternatives around entities
- A confident tone hiding a fragile probability margin