Scope Note These results reflect enforcement behavior against a defined attack corpus. They do not represent exhaustive coverage of all possible adversarial techniques.
What These Benchmarks Measure
These benchmarks evaluate governance enforcement outcomes, not model alignment. Each attack is considered “caught” when a deterministic policy decision (DENY or BLOCK) is returned by TealTiger’s enforcement layer.AIGoat OWASP LLM Top 10 Benchmark
AIGoat is an open-source AI security playground that provides hands-on attack labs covering the full OWASP Top 10 for LLM Applications. TealTiger’s benchmark extracts attack patterns from AIGoat’s labs and runs them through the full stack.Full Stack Results
27 attacks · 27 caught · 100% catch rateWhat Each Layer Catches
TealTiger uses defense in depth. Different components handle different attack classes:Guardrails-Only Baseline (v0.2.2)
For comparison, running the same attack corpus through only the basic guardrails (without TealEngine policies) produces a lower catch rate. This demonstrates the value of the full stack. 34 attacks · 18 caught · 52.9% catch rate
The jump from 52.9% to 100% is entirely due to TealEngine’s policy layer — tool policies, identity enforcement, code execution policies, and behavioral limits.
How to Interpret These Results
- Results are scenario‑based and limited to the tested corpus.
- Effectiveness depends on policy configuration and coverage.
- Benchmarks demonstrate enforcement behavior, not complete security assurance.
- TealTiger is compliance‑enabling and does not claim certification based on benchmark performance.
Attack Categories Tested
LLM01: Prompt Injection
- Direct instruction override (“ignore all previous instructions”)
- Context manipulation (delimiter injection, chat template injection)
- Indirect injection (foreign language translation tasks)
- Role-play jailbreaks (DAN, DUDE, persona assignment)
- Encoding attacks (base64-encoded payloads)
LLM02: Sensitive Information Disclosure
- PII in user input (SSN, credit cards, emails, phone numbers)
- Bulk data exfiltration requests
- Targeted PII extraction for specific users
- Training data completion attacks
LLM05: Improper Output Handling
- XSS payloads (
<script>tags,document.cookie) - SQL injection (
DROP TABLE,DELETE FROM) - OS command injection (
cat /etc/passwd, shell commands)
LLM06: Excessive Agency
- Unauthorized financial actions (large refunds without approval)
- Privilege escalation (admin access requests)
- Bulk data export to external destinations
- Destructive database operations (delete, drop)
LLM07: System Prompt Leakage
- Direct extraction (“print your system prompt”)
- Verbatim repetition attacks
- Social engineering via developer persona
LLM10: Unbounded Consumption
- Token exhaustion (extremely long output requests)
- Context window flooding (100K+ character inputs)
Running the Benchmarks
Both benchmark test suites are included in the TealTiger SDK:Methodology
- Attack prompts are inspired by AIGoat OWASP LLM Top 10 labs
- Each attack is run through the full TealTiger stack (TealGuard + TealEngine)
- A “catch” means at least one component returned a DENY decision
- Tests are deterministic and repeatable — no LLM inference involved
- Results are generated from automated test runs, not manual testing
Evidence and Auditability
Each benchmark execution corresponds to one or more recorded enforcement decisions produced by TealTiger’s policy engine. These decisions are captured as immutable evidence records, including:- trace identifiers linking decisions to benchmark runs
- evaluated policy and control identifiers
- deterministic allow/deny outcomes
- stable reason codes explaining enforcement decisions