RESEARCHInvestigateNEXT 12 MONTHS
The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
arXiv cs.CL — Computation and Language
Factual evidence
What the source reports
Research demonstrates LLM safety refusal behavior varies across random seeds and temperatures, invalidating single-shot safety evaluations.
Inspect the evidence
- Inclusion basis
- Enterprise AI
- Publisher and source type
- arXiv cs.CL — Computation and Language · RESEARCH
- Published by source
- 15 October 2026
- Collected by OneBench
- 31 Aug 2026, 19:26 UK
Stored source excerpt
arXiv:2512.12066v3 Announce Type: replace-cross Abstract: Current safety evaluations of large language models rely on single-shot testing, implicitly assuming that model responses are deterministic and representative…
Short excerpt from the collected text, not the full source. Use the source link to read it in context.
The factual summary is a OneBench synthesis, not a quotation or independent verification. Collection time is not publication time. Open the source for its full context; related reporting can share the same underlying announcement.