Method2026-05-31#evals
Red-team prompts with harmless mutants
Prompt boundaries are brittle when tests are too literal. A model may refuse the obvious bad phrasing and comply with the polite version, the translated version, or the version hidden inside a roleplay.
Mutants make that brittleness visible without adding new dangerous content. They perturb the wrapper, not the underlying safety category.
Use them for both sides of the boundary: prompts that should be refused and prompts that should remain allowed. Over-refusal is a product bug too.
Aha moment
A boundary that only holds for one phrasing is a memorized example, not a reliable rule.
Try this
Generate harmless variants that preserve intent while changing tone, order, indirection, language, and role framing.
Watch for
- Safety behavior that fails when the user sounds polite
- Allowed prompts that become over-refused after hardening
- Fixtures that mutate the danger instead of just the wrapper