How scenarios are generated
Skrut generates test scenarios from two sources:- Your agent’s knowledge base — grounded scenarios that probe for hallucinations and broken refusals.
- Red-team templates — adversarial patterns (prompt injection, jailbreaks, PII leaks) applied to your agent’s context.
Grading
Every turn in a conversation is graded on a 0–100 scale across:- Accuracy — did the agent answer correctly?
- Refusal — did it correctly refuse out-of-scope or harmful requests?
- Consistency — does it contradict itself across turns?