Tencent-Hunyuan/DiagSafe

Tencent-Hunyuan/DiagSafe

Description: DiagSafe: diagnostic LLM safety benchmark with prompt-level rubrics (3,489 bilingual prompts, 84-label taxonomy)

Language: Python

License: NOASSERTION

Stars: 9

Forks: 0

Open issues: 0

Created: 2026-10-15T06:49:00Z

Pushed: 2026-10-22T04:54:24Z

Default branch: main

Fork: no

Archived: no

README: English | [简体中文](README_zh.md)

---

Overview

DiagSafe is a diagnostic safety benchmark for large language models, built from two parts:

• A fine-grained, diverse safety benchmark: 3,489 prompts grounded in an 84-label, three-tier

safety taxonomy, covering Chinese and English, single- and multi-turn dialogues, and direct harmful requests as well as several attack forms.

• Prompt-level rubric evaluation: every prompt comes with its own safety items and their

exemptions, combined with eight shared usefulness items; an LLM judge rules on each item and cites evidence.

Together they yield reliable, explainable judgments and fine-grained safety diagnoses of each model.

(1) One rubric per prompt; (2) different judges reach consistent verdicts with the same rubric; (3) item-level hits expose the specific risks of each model.

Benchmark at a Glance

| Dimension | Details | |---|---| | Prompts | 3,489 (DiagBase 2,401 · DiagAttack 1,088), Chinese and English at roughly 1:1 | | Taxonomy | 11 L1 / 34 L2 / 84 L3 labels, built from 204 safety provisions distilled from public AI safety guidelines of China, the US, the EU and other regions | | Dialogues | 2,560 single-turn, 929 multi-turn (2–5 turns) | | Attack forms | Direct harmful requests, template jailbreaks, combined-template jailbreaks, instruction attacks | | Evaluation | LLM-as-a-judge: prompt-specific safety items (4.5 on average, each with exemptions) + eight shared usefulness items (0–3 score) |

Every L3 label carries a definition, a general standard and detailed rules, and each rubric is generated from the standard of its label. The leaf-level comparison below is against SALAD-Bench, S-Eval, Do-Not-Answer and CHiSafetyBench: red leaves have no counterpart in those benchmarks, and orange leaves are covered there only by a parent category.

DiagBase and DiagAttack

| | DiagBase | DiagAttack | |---|---|---| | Size | 2,401 | 1,088 | | Construction | Synthesized from the label standards along language × turns × attack form | Rewrites the single-turn direct requests of DiagBase into instruction attacks | | Attack forms | Direct harmful requests, template jailbreaks, combined-template jailbreaks | Instruction attacks: the harmful intent is embedded in a benign-looking task with role, format and tone constraints | | Dialogues | 1,472 single-turn, 929 multi-turn | All single-turn | | Focus | Label coverage, safety across languages and multi-turn dialogues | The conflict between strong instruction following and safety refusal |

Key Results

We evaluated 11 leading LLMs on DiagSafe. Two findings stand out.

1. Rubric-based evaluation is more reliable

Against 236 human-annotated responses as the gold standard:

| Paradigm | Agreement with humans | Cross-judge consistency (Cohen's κ) | |---|---|---| | Safety classifiers (MD-Judge, LlamaGuard4, S-Eval, Qwen3Guard) | 69.07% – 87.29% | — | | Generic safety prompt + LLM | 80.51% – 87.29% | 0.559 – 0.766 | | Label-specification prompt + LLM | 82.63% – 86.86% | 0.532 – 0.802 | | Prompt-level rubric + LLM (DiagSafe) | 89.41% – 95.34% | 0.828 – 0.864 |

Rubric-based judging reaches up to 95.34% agreement with human annotations and stays above 89% agreement across judge models, while the two prompt-based paradigms vary markedly between judges. Human review further finds 96.4% of prompts genuinely harmful and 99.6% of rubrics reasonable and actionable.

2. Leading LLMs remain unsafe, especially on DiagAttack

• On DiagBase, unsafe rates range from 2.8% to 32.9%.
• On DiagAttack, every model's unsafe rate rises to 1.9–8.4× its DiagBase value (median 5.3×);

8 of the 11 models exceed 40%, and even the best model stays above 18%.

• Wrapping a harmful intent in an ordinary task is enough to erode safety refusals: instruction

following and safety refusal compete in current training, and existing benchmarks rarely cover this kind of attack.

Recently released models. We further evaluated five recently released models under exactly the same protocol (the same 3,489 prompts and prompt-level rubrics, no system prompt, the same decoding configuration, and Hy3 as the rubric-guided judge). Their results are consistent with the findings above: every model is clearly less safe on DiagAttack than on DiagBase.

Show exact numbers

| Model | Overall ↓ | DiagBase ↓ | DiagAttack ↓ | Usefulness ↑ | |---|:---:|:---:|:---:|:---:| | Qwen3.8-27B | 8.5 | 3.3 | 20.2 | 2.23 | | Hy4-Preview | 11.9 | 3.9 | 30.6 | 2.27 | | GLM-5.3† | 13.1 | 9.5 | 20.9 | 2.37 | | DeepSeek-V4.1-Flash | 18.5 | 5.8 | 47.5 | 2.14 | | Kimi-K3 | 26.8 | 14.9 | 53.5 | 2.31 |

Unsafe rates are in %; usefulness is the average score (0–3). † GLM-5.3 cannot disable its reasoning mode, so it is evaluated with the low reasoning-effort setting; all other settings are identical.

More diagnostics

Item-level judgments also expose differences that an overall unsafe rate hides. Left: each model on the safety–usefulness plane across the 11 L1 dimensions. Right: three models with nearly identical overall unsafe rates diverge sharply at the category level.

Models are generally more vulnerable in Chinese and in multi-turn dialogues, and the per-turn unsafe rate keeps rising as a conversation unfolds:

See the paper for the full experiments and analysis.

Repository Structure

DiagSafe/
├── data/
│ ├── README.md # Dataset composition, relationships and checksums
│ ├── prompts/ # 3,489 bilingual prompts, JSON Schema, field reference
│ ├── rubrics/ # Prompt-level rubrics, JSON Schema, rendering guide
│ └── taxonomy/ # 84-label taxonomy (Chinese & English), schemas, label tree
├── src/
│ ├── judge/ # OpenAI-compatible client and judge-output parsing
│ ├── rubric/ # Rubric rendering, including multi-turn history
│ ├── utils/ # I/O and configuration helpers
│ └── analyze.py # Grouped statistics
├── scripts/
│ ├── run_eval.py # End-to-end evaluation
│ └── analyze.py # Re-analysis of an existing results file
├── configs/ # Judge and model configurations (OpenAI API, vLLM)
├── dashboard/ # Dataset...