LLM- und Guardrail-Evaluationen auf dem NVIDIA DGX Spark (GB10). Modell- und Guard-Namen sind anklickbar — Fall für Fall mit Prompt, Antwort und Judge. Rohdaten bleiben lokal; Sicherheits-Playbook (04) wird nicht publiziert.
Auf dem GB10 selbst serviert (vLLM), Judge-bewertet. Vollständiges Kohorten-Roster — jedes geplante Modell mit Status, nicht nur die bestandenen. 21 Modelle (✅ gültig 21). Gültige Modelle anklicken → Detail (Prompt, Antwort, Judge je Fall). Sicherheit (04) ausgeschlossen.
| Modell | Release | Gesamt | K.O. | Qualität | Deutsch | Bias | Code | Tok/s | TTFT | Lizenz |
|---|---|---|---|---|---|---|---|---|---|---|
| Muse-Glimmer-30B-NVFP4 ↗ | 2026-08 | K.O. 86% | 3 | 82% | 100% | 78% | 90% | 12.4 | 91 ms | apache-2.0 |
| Muse-Glimmer-30B ↗ | 2026-08 | K.O. 82% | 6 | 77% | 100% | 89% | 90% | 4.2 | 254 ms | apache-2.0 |
| Muse-Glimmer-30B-NVFP4-DFlash ↗ | 2026-08 | K.O. 78% | 5 | 71% | 100% | 89% | 90% | 8.5 | 245 ms | — |
| Gemma-4-31B ↗ | 2026-03 | K.O. 77% | 8 | 71% | 75% | 89% | 80% | 3.8 | 397 ms | apache-2.0 |
| Gemma-4-26B-A4B ↗ | 2026-03 | K.O. 76% | 11 | 69% | 100% | 89% | 90% | 23.0 | 92 ms | apache-2.0 |
| Ornith-1.0-35B-FP8 ↗ | 2026-06 | K.O. 76% | 13 | 66% | 100% | 100% | 80% | 30.8 | 105 ms | mit |
| Nemotron-3-Super ↗ | 2026-03 | K.O. 73% | 8 | 69% | 75% | 89% | 80% | 15.9 | 247 ms | nvidia-nemotron-open-model-license |
| Qwen3.8-27B-FP8 ↗ | 2026-08 | K.O. 73% | 9 | 71% | 50% | 78% | 70% | 7.9 | 181 ms | apache-2.0 |
| Nemotron-3.5-Lightning-30B-A3B ↗ | 2026-08 | K.O. 72% | 16 | 66% | 25% | 100% | 100% | 42.3 | 93 ms | openmdw-1.1 |
| Qwen3.6-27B-FP8 ↗ | 2026-04 | K.O. 64% | 14 | 63% | 0% | 100% | 30% | 7.5 | 242 ms | apache-2.0 |
| LFM2.5-8B-A1B ↗ | 2026-05 | K.O. 63% | 9 | 60% | 0% | 89% | 60% | 55.9 | 51 ms | lfm1.0 |
| Apertus-v1.5-8B ↗ | 2026-07 | K.O. 61% | 13 | 58% | 50% | 100% | 30% | 14.2 | 83 ms | apache-2.0 |
| Apertus-v1.5-70B ↗ | 2026-07 | K.O. 60% | 13 | 58% | 50% | 78% | 40% | 3.1 | 623 ms | apache-2.0 |
| Nemotron-3-Nano-30B ↗ | 2025-12 | K.O. 60% | 20 | 53% | 25% | 89% | 80% | 42.3 | 128 ms | nvidia-nemotron-open-model-license |
| Nemotron-Puzzle-75B ↗ | 2026-06 | K.O. 60% | 15 | 55% | 0% | 100% | 60% | 20.8 | 131 ms | openmdw-1.1 |
| Granite-4.1-30B ↗ | 2026-04 | K.O. 59% | 19 | 55% | 25% | 100% | 60% | 3.9 | 281 ms | apache-2.0 |
| Qwen3.6-35B-A3B-FP8 ↗ | 2026-04 | K.O. 58% | 15 | 55% | 0% | 100% | 30% | 17.9 | 154 ms | apache-2.0 |
| Mistral-Small-4 ↗ | 2026-03 | K.O. 55% | 23 | 50% | 75% | 78% | 60% | 31.4 | 94 ms | apache-2.0 |
| Olmo-3.1-32B-Instruct ↗ | 2025-12 | K.O. 54% | 19 | 42% | 0% | 100% | 60% | 3.7 | 342 ms | apache-2.0 |
| Ornith-1.0-9B ↗ | 2026-06 | K.O. 51% | 17 | 39% | 25% | 78% | 70% | 12.3 | 118 ms | mit |
| Apertus-v1.1-4B ↗ | 2026-05 | K.O. 30% | 31 | 21% | 25% | 89% | 10% | 27.6 | 42 ms | apache-2.0 |
Frontier-Modelle über LiteLLM als Referenzrahmen — ein Endpoint für Cloud- und lokale Modelle, von SouthByte empfohlen. Gleicher Testsatz, Judge claude-sonnet-5. Tok/s & TTFT messen hier Cloud+Netz (nicht lokale Hardware). 28 Modelle · Pass-Rate je Playbook. Modellname anklicken → Detail (Prompt, Antwort, Judge je Fall). Sicherheit (04) ausgeschlossen.
| Modell | Release | Gesamt | K.O. | Qualität | Deutsch | Bias | Code | Tok/s | TTFT | Lizenz |
|---|---|---|---|---|---|---|---|---|---|---|
| Grok-4.6 ↗ | 2026-08 | K.O. 93% | 1 | 95% | 100% | 89% | 90% | 18.3 | 715 ms | xAI · proprietär (API) |
| Claude-Sonnet-5 ↗ | 2026-06 | K.O. 91% | 1 | 92% | 100% | 78% | 90% | 98.4 | 1202 ms | Anthropic · proprietär (API) |
| Grok-4.5 ↗ | 2026-07 | K.O. 89% | 7 | 89% | 100% | 78% | 100% | 43.0 | 739 ms | xAI · proprietär (API) |
| Claude-Fable-5 ↗ | 2026-06 | PASS 88% | 0 | 92% | 100% | 89% | 60% | 76.7 | 2560 ms | Anthropic · proprietär (API) |
| Grok-4.1-Fast ↗ | 2025-11 | K.O. 88% | 3 | 87% | 100% | 89% | 90% | 101.2 | 676 ms | xAI · proprietär (API) |
| GPT-5.6-sol ↗ | 2026-07 | K.O. 87% | 6 | 82% | 100% | 100% | 100% | 54.9 | 1508 ms | OpenAI · proprietär (API) |
| GPT-5.6-terra ↗ | 2026-07 | K.O. 85% | 7 | 79% | 100% | 100% | 100% | 81.5 | 738 ms | OpenAI · proprietär (API) |
| GPT-5.6-luna ↗ | 2026-07 | K.O. 82% | 7 | 79% | 75% | 89% | 100% | 143.7 | 1007 ms | OpenAI · proprietär (API) |
| Claude-Opus-5 ↗ | 2026-07 | WARN 81% | 0 | 79% | 100% | 78% | 90% | 125.4 | 2921 ms | Anthropic · proprietär (API) |
| Kimi-K3 ↗ | 2026-07 | K.O. 80% | 2 | 77% | 100% | 78% | 90% | 43.0 | 1517 ms | proprietär (API) |
| MiMo-v2.5-Pro ↗ | 2026-04 | K.O. 78% | 6 | 74% | 75% | 89% | 90% | 48.8 | 1572 ms | proprietär (API) |
| Qwen3.7-Plus | 2026-06 | K.O. 78% | 13 | 73% | 100% | 89% | 90% | 59.4 | 689 ms | proprietär (API) |
| Gemini-3.1-Pro ↗ | 2026-02 | K.O. 75% | 9 | 71% | 100% | 78% | 90% | — | — | Google · proprietär (API) |
| GLM-5.2 ↗ | 2026-06 | K.O. 74% | 14 | 69% | 100% | 78% | 90% | 102.0 | 805 ms | proprietär (API) |
| MiniMax-M3 ↗ | 2026-06 | K.O. 74% | 8 | 71% | 100% | 67% | 90% | 157.9 | 1346 ms | proprietär (API) |
| Nemotron-3-Ultra-550B ↗ | 2026-06 | K.O. 73% | 15 | 68% | 100% | 78% | 90% | 23.5 | 1208 ms | proprietär (API) |
| DeepSeek-V4-Pro ↗ | 2026-04 | K.O. 72% | 14 | 66% | 100% | 78% | 90% | 42.7 | 2090 ms | proprietär (API) |
| Gemini-3.5-Flash-Lite ↗ | 2026-07 | K.O. 72% | 15 | 66% | 100% | 78% | 90% | 311.9 | 466 ms | Google · proprietär (API) |
| Gemini-3.6-Flash ↗ | 2026-07 | K.O. 72% | 15 | 66% | 100% | 67% | 100% | — | — | Google · proprietär (API) |
| Hunyuan-3 | 2026-07 | K.O. 68% | 15 | 63% | 100% | 67% | 90% | 61.9 | 1518 ms | proprietär (API) |
| MiMo-v2.5 ↗ | 2026-04 | K.O. 67% | 14 | 63% | 75% | 78% | 80% | 95.3 | 1019 ms | proprietär (API) |
| Qwen3.8-Max | 2026-08 | K.O. 67% | 7 | 66% | 50% | 89% | 60% | 47.4 | 998 ms | proprietär (API) |
| Claude-Haiku-4.5 ↗ | 2025-10 | K.O. 66% | 8 | 65% | 50% | 78% | 70% | 115.7 | 620 ms | Anthropic · proprietär (API) |
| Magistral-Medium ↗ | 2025-09 | K.O. 56% | 23 | 50% | 100% | 67% | 70% | 127.3 | 240 ms | Mistral · proprietär (API) |
| Mistral-Large ↗ | 2025-12 | K.O. 56% | 23 | 50% | 100% | 78% | 60% | 56.0 | 337 ms | Mistral · proprietär (API) |
| Mistral-Medium ↗ | 2026-04 | K.O. 55% | 22 | 47% | 75% | 78% | 80% | 65.5 | 249 ms | Mistral · proprietär (API) |
| Step-3.7-Flash ↗ | 2026-05 | K.O. 48% | 22 | 44% | 50% | 78% | 50% | 122.4 | 1748 ms | proprietär (API) |
| Ministral-8B ↗ | 2024-10 | K.O. 38% | 28 | 31% | 0% | 78% | 60% | 137.5 | 180 ms | Mistral · proprietär (API) |
Guard-Name anklicken → Fall für Fall (Wahrheit vs. Vorhersage). Kein Judge — das Label ist die Wahrheit.
| Guard | Release | Recall | FPR | Trap-FPR | Präz. | F1 | Acc | Lat ø | Lizenz |
|---|---|---|---|---|---|---|---|---|---|
| Granite-Guardian | 2026-04 | 0.974 | 0.023 | 0.042 | 0.974 | 0.974 | 0.976 | 1044.900 | apache-2.0 |
| Nemotron-3 | 2026-03 | 1.000 | 0.045 | 0.083 | 0.950 | 0.974 | 0.976 | 493.200 | nvidia-nemotron-open-model-license |
| Nemotron-3.5 | 2026-05 | 1.000 | 0.045 | 0.083 | 0.950 | 0.974 | 0.976 | 562.900 | openmdw-license-agreement |
| gpt-oss-safeguard | 2025-09 | 1.000 | 0.000 | 0.000 | 1.000 | 1.000 | 1.000 | 1566.800 | apache-2.0 |
| Shieldstral | 2026-07 | 1.000 | 0.023 | 0.042 | 0.974 | 0.987 | 0.988 | 44.300 | apache-2.0 |