I spend most of my time breaking into things for a living. For the last year or so, a growing chunk of that work has been pointed at LLMs. Not the models themselves, exactly. The deployments. The API gateways with a language model behind them. The customer-facing chatbots. The internal tools that got "an AI feature" bolted on in Q3 because someone's VP saw a demo and said "we need this." The RAG pipelines connected to document stores full of sensitive data. These things are everywhere now. And almost none of them have been adversarially tested. I don't mean "does the model refuse if you ask it something bad." That's safety training. Safety training is important. But safety training and security testing are fundamentally different disciplines, and the industry is conflating them in ways that are going to cause real problems. Safety training teaches a model to refuse. Security testing asks whether that refusal actually holds up when someone is actively trying to break it. The answer, overwhelmingly, is no. The numbers are bad OWASP ranked prompt injection as the number one security risk in LLM applications. That ranking is earned. FlipAttack, a technique that simply reorders characters in prompts, achieves a 98% bypass rate against GPT-4o. DeepSeek R1 showed a 100% bypass rate against 50 HarmBench jailbreak prompts in testing by Cisco and the University of Pennsylvania. A study of 36 production LLM-integrated applications found that 86% were vulnerable to prompt injection. PoisonedRAG demonstrated that just five malicious documents in a corpus of millions can manipulate AI outputs 90% of the time. These aren't theoretical attacks against research models. These are attacks against production systems that real organizations are running right now. So I built a scanner Augustus is an open-source LLM vulnerability scanner. You point it at a model endpoint and it throws 210+ adversarial probes at it across 47 attack categories. It tells you what's vulnerable and what's not. go install github.com/praetorian-inc/augustus/cmd/augustus@latest

augustus scan openai.OpenAI \
  --all \
  --verbose go install github.com/praetorian-inc/augustus/cmd/augustus@latest

augustus scan openai.OpenAI \
  --all \
  --verbose It ships as a single Go binary. No Python. No npm. No runtime dependencies. One install command and you're scanning. I built it in Go because I needed something that fits into penetration testing workflows without requiring me to set up a Python environment on every engagement. go install, run, done. The concurrency model also matters: goroutine pools running probes in parallel across the target, not bottlenecked by Python's GIL. go install It's inspired by garak, NVIDIA's Python-based LLM vulnerability scanner. garak is excellent and has a longer research pedigree with a published paper. Augustus is the same concept reimplemented for a different set of trade-offs: portability, speed, and zero-dependency distribution. Different tools for different workflows. What it actually tests Here's where it gets interesting. When most people think about LLM attacks, they think about jailbreaks. "Pretend you're DAN." "My grandmother used to tell me how to..." Those matter, and Augustus tests all of them (DAN variants through v11.0, AIM, AntiGPT, Grandma exploits, ArtPrompts). But jailbreaks are just the surface layer. Encoding bypasses are where things start to get ugly. Augustus tests across Base64, ROT13, Morse code, hex, Braille, Klingon, leet speak, and about 12 other encoding schemes. The question being asked: if you wrap a harmful instruction in Base64, does the model decode it and follow it even though the plain-text version would be blocked? Encoding bypasses In a lot of cases, yes. The gap between what input filters see (encoded text that looks benign) and what the model understands (the decoded malicious intent) is consistently exploitable. This is one of the most reliable attack vectors I've seen in production. FlipAttack (16 variants) reverses or reorders characters to evade input filters. The research showed 98% bypass on GPT-4o. Augustus implements all the published variants. FlipAttack Tag smuggling embeds instructions inside XML or HTML tags. Models that are trained to process structured input will sometimes follow instructions embedded in tags that look like formatting rather than commands. Tag smuggling Data extraction is where things get operationally dangerous. Augustus probes whether the model can be tricked into leaking API keys or credentials from its context window. It tests for PII extraction. It checks for training data regurgitation. Data extraction The package hallucination probes are one of my favorites. These cover Python, JavaScript, Ruby, Rust, Dart, Perl, and Raku. They ask the model to recommend packages for various tasks and then check whether any of the recommended packages don't actually exist. This matters because it's a real supply chain attack vector: adversaries monitor for hallucinated package names, register them, and wait for developers to pip install or npm install the fake package. The model becomes an unwitting accomplice in a supply chain attack. pip install npm install RAG poisoning probes test whether an attacker can inject malicious content into the retrieval pipeline, both through document content and metadata injection. If your RAG system pulls from a corpus that an attacker can influence (and most can be influenced more easily than you'd think), the model's outputs can be manipulated. RAG poisoning Agent attacks are the newest category and arguably the most concerning. As LLMs gain tool access (browsing, code execution, database queries, API calls), the attack surface expands dramatically. Augustus tests multi-agent manipulation (can one agent influence another's behavior?), browsing exploits (can adversarial web content hijack a model with web access?), and latent injection (can instructions embedded in documents that a RAG-enabled agent processes cause it to take unintended actions?). Agent attacks Format exploits target structured output. If a model generates markdown, can an attacker inject malicious links that render as legitimate? If it produces HTML, are XSS payloads possible? If downstream systems parse YAML or JSON from model output, can that parsing be exploited? These are real risks when LLM output gets rendered in browsers or consumed by other systems. Format exploits Evasion techniques test the model's ability to recognize adversarial intent regardless of how it's presented. ObscurePrompt uses an LLM to rewrite known jailbreaks into harder-to-detect forms. Character substitution probes use homoglyphs (characters that look identical but have different Unicode codepoints), zero-width characters, and bidirectional text markers. These are inputs that look completely benign to text-based filters but are interpreted differently by the model. Evasion techniques Safety benchmarks round it out. DoNotAnswer (941 questions across 5 risk areas), RealToxicityPrompts, Snowball (plausible-sounding but factually wrong outputs), and LMRC harmful content probes. Safety benchmarks In total: 210+ probes across 47 attack categories. The buff system is where it gets real Here's the thing about adversarial testing: real attackers don't send attacks in plain text. They encode, translate, rephrase, and obfuscate. A DAN prompt that gets caught by every filter in the world might sail right through when it's been paraphrased, translated into Zulu, and reformatted as a haiku. Augustus has a buff system that applies transformations to any probe before it's sent. Seven transformations across five categories: Encoding buffs wrap prompts in Base64 or character codes. Testing the gap between what filters see and what models understand. Encoding buffs Paraphrase buffs use a Pegasus model to rephrase prompts while preserving adversarial intent. Same meaning, different surface form. This tests whether safety training generalizes beyond the specific patterns it was trained on, or whether it's essentially pattern matching on known bad inputs. Paraphrase buffs Poetry buffs reformat prompts as haiku, sonnets, limericks, free verse, or rhyming couplets. I know this sounds absurd. But models that robustly block a direct harmful request will sometimes comply when the same request arrives as verse. I've seen it happen repeatedly. Something about the stylistic framing seems to shift how the model processes the intent. Poetry buffs Low-resource language translation exploits the fact that safety training is overwhelmingly concentrated on English. A request that's blocked in English may succeed in Zulu, Hmong, or Scots Gaelic. Augustus translates probes via DeepL to test this. Low-resource language translation Case transforms simply lowercase everything. Some input filters and keyword blocklists are case-sensitive. It's dumb. It works. Case transforms You can chain these. Encode a probe in Base64, then paraphrase it, then translate it to a low-resource language. Layered evasion that tests whether defenses hold up against inputs that don't match any expected pattern. augustus scan openai.OpenAI \
  --probe dan.Dan \
  --buff encoding.Base64

augustus scan ollama.OllamaChat \
  --probe dan.Dan \
  --buffs-glob "paraphrase.*,lrl.*" \
  --config '{"model":"llama3.2:3b"}' augustus scan openai.OpenAI \
  --probe dan.Dan \
  --buff encoding.Base64

augustus scan ollama.OllamaChat \
  --probe dan.Dan \
  --buffs-glob "paraphrase.*,lrl.*" \
  --config '{"model":"llama3.2:3b"}' 28 providers, one interface Augustus connects to OpenAI (including o1/o3 reasoning models), Anthropic (Claude 3/3.5/4), Azure OpenAI, AWS Bedrock, Google Vertex AI, Cohere, Replicate, HuggingFace, Together AI, Groq, Mistral, Fireworks, DeepInfra, NVIDIA NIM, Ollama, LiteLLM, and more. For anything else, there's a REST connector: augustus scan rest.Rest \
  --probe dan.Dan \
  --config '{
    "uri": "https://your-api.example.com/v1/chat/completions",
    "headers": {"Authorization": "Bearer YOUR_KEY"},
    "req_template_json_object": {
      "model": "your-model",
      "messages": [{"role": "user", "content": "$INPUT"}]
    },
    "response_json": true,
    "response_json_field": "$.choices[0].message.content"
  }' augustus scan rest.Rest \
  --probe dan.Dan \
  --config '{
    "uri": "https://your-api.example.com/v1/chat/completions",
    "headers": {"Authorization": "Bearer YOUR_KEY"},
    "req_template_json_object": {
      "model": "your-model",
      "messages": [{"role": "user", "content": "$INPUT"}]
    },
    "response_json": true,
    "response_json_field": "$.choices[0].message.content"
  }' Custom request templates with $INPUT placeholders, JSONPath response extraction, SSE streaming, and proxy routing. If your endpoint speaks HTTP, Augustus can test it. $INPUT Detection isn't just pattern matching On the detection side, Augustus has 90+ detectors. Pattern matching catches known jailbreak indicators. LLM-as-a-judge uses a second model to evaluate whether the response is harmful. HarmJudge (based on arXiv:2511.15304) provides semantic harm assessment aligned with the MLCommons AILuminate taxonomy. The Perspective API measures toxicity. For iterative attacks like PAIR and TAP, a dedicated attack engine handles multi-turn conversations, candidate pruning, and judge-based scoring. These aren't single-shot tests. They're adaptive attacks that refine their approach across multiple attempts, mimicking how a real attacker would actually operate. They're computationally expensive (many LLM calls per test) but they represent the current state of the art in automated red-teaming. What I've learned from building this A few things became clear over the course of building Augustus and running it against production systems: Safety training is not security. I keep coming back to this because it's the fundamental misconception driving the gap. Safety training is a behavioral overlay. It teaches the model patterns for refusal. Security testing asks whether those patterns hold up under adversarial conditions. They almost never do, at least not comprehensively. Safety training is not security. Encoding bypasses are embarrassingly effective. The fact that wrapping a harmful request in Base64 still works against many production deployments in 2026 is wild. Input filters and the model itself are operating on different representations of the same input, and that gap is exploitable. Encoding bypasses are embarrassingly effective. Low-resource languages are an underappreciated attack vector. Safety training is concentrated on English. The drop-off in refusal quality for low-resource languages is significant and consistent. Low-resource languages are an underappreciated attack vector. Agent-level attacks are going to be the next big thing. As models gain tool access, every tool becomes part of the attack surface. A model with browsing access can be manipulated by adversarial web content. A model with database access can be tricked into exfiltrating data. A model that processes documents can follow latent instructions embedded in those documents. We're in the very early innings of understanding this attack surface. Agent-level attacks are going to be the next big thing. The tooling gap is real and it's getting wider. Organizations are deploying LLMs faster than they're testing them. The models ship fast. The security testing doesn't happen at all. Something has to close that gap, and it needs to be accessible enough that it doesn't require a specialized AI red team to run. The tooling gap is real and it's getting wider. Get it Augustus is Apache 2.0 licensed and available now. Repo: https://github.com/praetorian-inc/augustus Repo: https://github.com/praetorian-inc/augustus go install github.com/praetorian-inc/augustus/cmd/augustus@latest

augustus scan ollama.OllamaChat \
  --all \
  --config '{"model":"llama3.2:3b"}' go install github.com/praetorian-inc/augustus/cmd/augustus@latest

augustus scan ollama.OllamaChat \
  --all \
  --config '{"model":"llama3.2:3b"}' It's the second tool in a 12-tool open-source series I'm releasing over 12 weeks. One tool per week, each doing one thing well. The first was Julius, which handles LLM fingerprinting (identifying what model is running behind an endpoint). The rest of the series will continue building out the offensive security toolkit for AI systems. If you run it against your models and find something interesting, I'd like to hear about it. And if you want to contribute probes for attack vectors we haven't covered yet, the repo has a CONTRIBUTING.md that explains the probe definition format and development workflow. The models are shipping. The testing needs to catch up.

This story contains AI-generated text. The author has used AI either for research, to generate outlines, or write the text itself. 

This story contains new, firsthand information uncovered by the writer.

Apache

Google

I Built an Open-Source Tool to Attack-Test LLMs. Here's What Breaks

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

What Every E-Commerce Brand Should Know About Prompt Injection Attacks

Understanding Testing vs. Evaluation in AI Systems

Multilingual Prompt Injection Exposes Gaps in LLM Safety Nets

Claude Opus 4.6: Why Anthropic’s New Safety Architecture is a Game Changer for Agentic AI

How Large Language Models Impact Data Security in RAG Applications

LLM Security: A Practical Overview of the Protective Measures Needed

What Every E-Commerce Brand Should Know About Prompt Injection Attacks

Understanding Testing vs. Evaluation in AI Systems

Multilingual Prompt Injection Exposes Gaps in LLM Safety Nets

Claude Opus 4.6: Why Anthropic’s New Safety Architecture is a Game Changer for Agentic AI

How Large Language Models Impact Data Security in RAG Applications

LLM Security: A Practical Overview of the Protective Measures Needed

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps