{
  "ok": true,
  "data": {
    "entry": {
      "id": "token-counter",
      "kind": "tool",
      "kindLabel": "Tool",
      "title": "Keep a tokenizer open in a tab",
      "body": "Half of weird model behavior is a tokenization story. ' apple' and 'apple' are different tokens. Numbers split in unintuitive ways. Pasting your prompt into a tokenizer viewer takes ten seconds and explains a surprising number of bugs.",
      "tag": "tooling",
      "date": "2025 · 09 · 03",
      "publishedAt": "2025-09-03",
      "note": null,
      "insight": {
        "aha": "The model never sees your characters. It sees chunks, spaces, merges, and weird number splits.",
        "tryThis": "Paste the exact prompt, including spaces and separators, into a tokenizer before debugging prompt behavior.",
        "watchFor": [
          "Leading spaces changing label tokens",
          "Numbers split into surprising chunks",
          "Few-shot examples with inconsistent separators"
        ]
      },
      "extended": [
        "Leading spaces, casing, and digit grouping all change how text is split, and the model only ever sees the tokens, never your characters. A trailing space can silently sabotage a few-shot format.",
        "When something is off and you cannot explain it, paste the exact string into a tokenizer view before you theorize. The answer is in there more often than you would expect."
      ],
      "url": "https://tips.platphormnews.com/entries/token-counter"
    }
  }
}