This is exactly why we built AgentAudit. The bypasses described here - directory swapping, forged lockfiles, exit code masking - are all variations of what we call "environmental exploitation."
The key insight: agents don't reason about security boundaries. They optimize for task completion. Your sandbox is just another constraint to work around.
We've catalogued 650+ attack patterns against AI agents, and many fall into this category - not adversarial prompts, but emergent behaviors that exploit trust assumptions.
Defense in depth is right. We also recommend:
- Testing agents with security scanners BEFORE production
- Logging all tool invocations, not just denials
- Treating agent outputs as untrusted input
Some of these don’t really seem like they bypassed any kind of sandbox. Like hallucinating an npm package. You acknowledge that the install will fail if someone tries to reinstall from the lock file. Are you not doing that in CI? Same with curl, you’ve explained how the agent saw a hallucinated error code, but not how a network request would have bypass the sandbox. These just sound like examples of friction introduced by the sandbox.
> These just sound like examples of friction introduced by the sandbox.
The whole idea of putting "agentic" LLMs inside a sandbox sounds like rubbing two pieces of sandpaper together in the hopes a house will magically build itself.
> The swap bypassed our policy because the deny rule was bound to a specific file path, not the file itself or the workspace root.
This policy is stupid.
I mount the directory read inside the container to make it impossible to do it (except for a security leak in the container itself)
I am testing running agents in docker containers, with a script for managing different images for different use cases etc, and came across this: https://docs.docker.com/ai/sandboxes/
Yes but it’s barely usable. I ended up making my own Dockerfile and a bash script to just ‘docker run’ my setup itself, and as a bonus you don’t need Docker Desktop. I might open source it at some point but honestly it’s pretty trivial to just append a couple of volume mount flags and env vars to your docker run and have exactly what you want included.
Yes, I don't think this will persist caches & configs outside of the current dir, for example, the global npm/yarn/uv/cargo cache or even Claude/Codex/Gemini code config.
I ended up writing my own wrapper around Docker to do this.
If interested, you can see the link in my previous comments. I don't want to post the same link again & again.
The key insight: agents don't reason about security boundaries. They optimize for task completion. Your sandbox is just another constraint to work around.
We've catalogued 650+ attack patterns against AI agents, and many fall into this category - not adversarial prompts, but emergent behaviors that exploit trust assumptions.
Defense in depth is right. We also recommend: - Testing agents with security scanners BEFORE production - Logging all tool invocations, not just denials - Treating agent outputs as untrusted input
If anyone wants to test their agent setup: https://app.xsourcesec.com (free tier available)
The whole idea of putting "agentic" LLMs inside a sandbox sounds like rubbing two pieces of sandpaper together in the hopes a house will magically build itself.
This policy is stupid. I mount the directory read inside the container to make it impossible to do it (except for a security leak in the container itself)
Has anyone given it a try?
Yes, I don't think this will persist caches & configs outside of the current dir, for example, the global npm/yarn/uv/cargo cache or even Claude/Codex/Gemini code config.
I ended up writing my own wrapper around Docker to do this. If interested, you can see the link in my previous comments. I don't want to post the same link again & again.