{
  "ok": true,
  "data": {
    "entry": {
      "id": "benchmark-time-horizon",
      "kind": "method",
      "kindLabel": "Method",
      "title": "Benchmarks are maps of failure, not trophies",
      "body": "Modern evals are getting harder because frontier models keep saturating old tests. Treat benchmarks as maps of what still breaks, especially long tasks, coding repairs, and questions that punish overconfidence.",
      "tag": "evals",
      "date": "2026 · 06 · 30",
      "publishedAt": "2026-06-30",
      "note": null,
      "insight": {
        "aha": "The most valuable benchmark output is often the broken example you can turn into a local fixture.",
        "tryThis": "Take one public benchmark-style failure and convert it into a small product-specific eval with expected safe behavior.",
        "watchFor": [
          "General scores used as production guarantees",
          "Short-answer tests used for long-running agents",
          "Benchmark examples ignored after the headline score"
        ]
      },
      "extended": [
        "A benchmark score can be useful, but the examples behind the score are usually more useful. They show what kind of task the model mishandled and what local fixture you should add before shipping.",
        "Long-task evals are especially important for agents because short answers hide drift. A model may solve one step and still lose the goal after thirty minutes of tool use, retries, and partial failures.",
        "The AHA is that a public benchmark should become a local test, not a marketing badge."
      ],
      "url": "https://tips.platphormnews.com/entries/benchmark-time-horizon"
    }
  }
}