Skip to main content
← All posts
researchBen Gilat

Digging into OpenAI's cyber security guardrail: what gets blocked and what doesn't

We replayed blocked requests against the real model to learn when the block lands, what causes it, and what makes no difference.


How we found out

Our agents read vulnerable open-source packages for a living. As part of their job, our agents download the package in a sandbox, read the code, write a PoC, and decide under what conditions the vulnerability is exploitable. They run on gpt-6.1-sol.

Every agent doing security work on a frontier model meets the model's safety classifier. Ours see it as one error: "This request has been flagged for potentially high-risk cyber activity." It normally stops about one thread in twenty. On November 5 that rose to one in five, so we decided to study it properly.

Blocked threads went from about 5% to about 20% of a day

Share of agent threads that hit the cyber block on gpt-6.1-sol, by day the thread started

5.8%
Oct 27
5%
Oct 28
4%
Oct 29
2.9%
Nov 1
4%
Nov 2
7.6%
Nov 3
6%
Nov 4
19.2%
Nov 5
20.8%
Nov 6
7.7%
Nov 8
18.5%
Nov 9
Production traces, Oct 27 to Nov 9, 2026. Low-volume days left out.

A block matters to anyone building on these models! The request fails, and the run either is downgraded to a worse model or dies. Without a harness built for security work, that happens silently. So we decided to measure the classifying system.

What we collected

We sampled 2k blocked threads from the last 14 days of production traces, and stripped them of sensitive data (anonymized).

For each one we kept the exact request that was refused, meaning the full conversation the model saw at that moment. We added an equal number of threads that were never blocked, cut at the same step count.

Then we did the thing that settles arguments: we replayed blocked requests against the real model and classifier, unchanged and with one thing altered at a time.

It is mostly about what the package does

The first surprise was how uneven the blocks are. The agent, the prompt and the tools are the same for every package, yet some packages are blocked in most runs and most are never blocked at all.

Two WebSocket packages are blocked in 3 of 4 runs; most packages almost never

Share of threads blocked. Only the 22 most-analysed of 560 packages are drawn. Dark blue marks network-facing packages.

socket.io-parser
78.1%
websocket-driver
72.5%
body-parser
35.6%
tar
35.3%
qs
32.8%
minimist
21.4%
postcss
17.1%
undici
10.8%
tough-cookie
10.6%
js-yaml
6.3%
axios
6%
lodash
3.8%
fast-xml-parser
3.5%
minimatch
2.2%
brace-expansion
1%
form-data
0.8%
node-forge
0.8%
jackson-databind
0.4%
proxy-addr
0%
log4j
0%
poi
0%
log4net
0%
Package-analysis threads, Oct 27 to Nov 10, 2026. All versions of a package pooled.

The packages at the top are network-facing. We labelled every package by name as network-facing or not (WebSocket and HTTP libraries, request parsers, cookies, proxies, URL handling) and compared the two groups:

GroupPackagesBlocked
Network-facing11516.1%
Everything else4456.2%

Network-facing packages are blocked 2.6 times as often. Threads on the same package are not independent, so we tested at the package level: resampling whole packages gives a 95% interval of 1.2 to 5.1 for that ratio, and shuffling the labels across packages gives p = 0.011.

Being on the network is not enough; parsing an attacker-controlled wire protocol seems to be.

The exceptions point the same way: tar and minimist are not network code, but their vulnerabilities are path traversal and prototype pollution, which are exploited the same way: by feeding the package a specially crafted input.

The same request does not get the same answer

We took 78 requests that were blocked in production and sent each one again, byte for byte, several times.

33 of 78 blocked requests were never blocked again

How often each of 78 blocked requests was blocked again in repeated unchanged replays

33
Never
16
A third or less
11
About half
10
Most replays
8
Every replay
Unchanged replays against gpt-6.1-sol, Nov 10, 2026. 78 threads, 296 replays.

Of 296 unchanged replays, 115 were blocked: 39%. For 33 threads the block never came back. Only 8 were blocked every time.

The model samples a different continuation on each call, and the classifier judges the continuation. A block is therefore a probability that depends on the conversation so far, not a fixed property of the request.

It also has a plain operational consequence: a blocked request is worth sending again before giving up on the model. With three retries, 84% (!) of blocked requests got through!

The verdict comes after the thinking and before the action

We expected a filter on the prompt: send something suspicious, get rejected at once. That is not what happens. We streamed the replays and timed every event.

The block lands mid-reply, typically 20 to 40 seconds in

Seconds from sending the request to the block, 239 blocked replies

4
Under 10 s
50
10 to 20 s
86
20 to 30 s
62
30 to 40 s
26
40 to 50 s
7
50 to 60 s
4
Over 60 s
Streamed replays against gpt-6.1-sol, Nov 10, 2026. 239 blocked replies.

The request is accepted and the model starts working. The block arrives a median of 27 seconds later, and what had streamed out before the block tells the story:

Streamed before the blockReplies
Nothing62
Reasoning only138
Reasoning and the first words of a text message39
A tool call0

So the classifier is not judging our prompt alone. It lets the model think, and it judges what the model is about to do, holding the action back until it has decided.

Thinking less does not help

If the verdict follows the reasoning, a natural guess is that less reasoning means fewer blocks.

So we reran the most consistently blocked threads with reasoning effort set to low. On the same seven threads they were blocked 86% of the time at low effort, against 75% at high effort. The block only arrived sooner, at a median of 22 seconds. Reasoning effort makes no difference.

What actually causes the block

We took the 27 threads that were blocked most consistently and changed one thing at a time.

Telling the agent to stop testing cut blocks from 70% to 10%

Share of replays blocked on 27 consistently blocked threads. Rows marked * were run on the first 7 threads only.

Unchanged
70.1%
Low reasoning effort *
85.7%
Last step removed *
60%
Everything the agent read blanked *
57.1%
Read-only tools
34.3%
Everything the agent wrote blanked
25%
Note: no new exploit code
23.3%
Report call forced
14.9%
Note: stop testing, report now
10.4%
Replays against gpt-6.1-sol, Nov 10, 2026. 27 threads from 17 packages, 2 to 4 replays per cell.

Three things stand out.

  • What the agent is about to do matters most. One added line telling it to stop testing and report took blocks from 70% to 10%. A softer line that only bans new exploit code gave 23%.
  • What the agent wrote matters more than what it read. Blanking its own test code cut blocks to 25%; blanking everything it had read left them at 57%.
  • The last step is not special. Removing it left blocks at 60%.

The logs agree: blocked threads were five times as likely to have the word "payload" in the agent's own writing (26% against 5%). But the word itself is not the cause: when we replaced "payload" with "input" everywhere, the block rate did not move.

What a security harness has to handle

  1. It depends on the package. Packages that parse a wire protocol are blocked at many times the base rate; most packages never are.
  2. A block is a probability. Four in ten blocked requests never fail again on replay, so one retry is cheap and often enough.
  3. The classifier judges the next action in context. The verdict lands after the reasoning and before any tool call, and the same history is blocked 70% or 10% of the time depending on what the agent is steered to do next.
  4. The agent's own exploit code is what raises the risk. What it read hardly matters.
  5. Reasoning effort and the last step make no difference.

This is more evidence that security agents need a harness around the model. Without that layer, the classifier decides which model does your work.

Note: This could not have been done without the help of Taxi, our tool for analyzing agent trajectories.