Hot take: 'the tests pass' is not a security check, and a new preprint shows why

What they found (all from the paper's abstract and text):

• Context built for functional correctness raised the functional pass rate by up to 8.2 percentage points, but the secure pass rate by at most 3.7.
• Withholding a safeguard (such as an authorization check) lowered the secure pass rate by up to 8.1 points.
• Inserting one insecure example raised correct-but-exploitable code by 6.3 points, even with the safeguard still present.
• Coding agents observed the relevant safeguard in only 27-36% of their runs. In the paper, 71% of agent search queries named functional things such as a file format and only 9% named security concepts.
• In the paper's agent table, Claude Code with Sonnet 4.5 passed 74.0% of functional checks and 18.4% of secure checks, and 55.6% of its generations passed every test but were still exploitable.

Why it matters. The code that gets merged is the code that passes tests. If an agent never looks at the check that protects an endpoint, it can write a working endpoint without it. The paper's pattern is that agents and retrieval tools search for what code does, not for the rule that constrains it.

Our verdict. Treat green tests as the floor. Before merging agent-written code that touches auth, parsing or user input, find the project's existing guard yourself and confirm the new code uses it. Agent scaffolds that go looking for safeguards would be worth trying; the paper suggests the direction but does not validate it.

Caveats. This is a preprint and has not been peer reviewed. The tasks ask a model to rewrite a masked region of real vulnerable code, and "secure" means the specific exploit failed, which does not rule out other flaws. Six projects supply 67 of the 145 tasks, and the authors say differences below about 5.6 points may go undetected. The agent figures come from restricted runs with a step and cost cap. Some of the context effects are not statistically significant after correction, and the authors say so.

Disclosure: MODEL did not run or test this study; every figure comes from the authors' paper at arxiv.org/abs/2611.06294, which we read in full text. Not sponsored. No affiliate links.

MODEL Editors