CAITLYN lets agent defenses synthesize their own countermeasures

Static defenses against prompt injection cannot survive novel attack vectors. If your defense cannot synthesize its own countermeasures when new patterns appear, your agent runtime is already compromised.

Prompt injection attacks against LLM agents manipulate external text retrieved during execution, forcing models to run harmful actions outside their intended scope. Current defenses hit a persistent trilemma: balancing runtime efficiency, contextual precision, and adaptability. Heavy LLM evaluators run up high token costs, while static rules fail the moment an attacker mutates their delivery payload.

CAITLYN addresses this as an agent-agnostic defense middleware split into two coordinated systems.

System I handles immediate defense against known threats using a two-tiered library. Tier-0 executes rule-based detection scripts for zero-token filtering. For inputs that require deeper analysis, Tier-1 runs optimized LLM-based inference to maintain contextual precision. On standard benchmarks, this tiered architecture matches the detection performance of state-of-the-art defenses while cutting token overhead compared to LLM-as-a-judge baselines.

System II acts as the adaptation engine. It continuously monitors for abnormal signals and autonomously synthesizes new, verified defense capabilities. When evaluated on the Emerging benchmark—which focuses on novel injection delivery techniques—static baselines and standalone System I configurations remain vulnerable. Adding System II closes that gap, substantially lowering the attack success rate across three diverse agent environments by producing verified countermeasures dynamically.

Where it breaks:
System I in isolation cannot handle novel injection techniques; running without System II leaves the runtime vulnerable to any payload not already codified in Tier-0 or Tier-1. Additionally, the entire adaptation loop hinges on the system's ability to monitor abnormal signals and verify new defenses. If abnormal signals go undetected, the synthesis pipeline cannot trigger.

The takeaway for builders shipping agent architectures: stop treating injection defenses as static rulebooks. Use tiered execution to keep runtime token overhead low, but build an autonomous loop that monitors anomalous executions and synthesizes verified defenses directly into the detection stack.