Claude Opus 5.5: An evaluation review and metrics benchmarks

In comparison to Claude Opus 5, Claude Opus 5.5 produced fewer blocker-level bugs, vulnerabilities, and code smells, and did so by writing significantly fewer lines of code and consuming fewer output tokens. This is one of Anthropic's cleanest, most efficient models.

Claude Opus 5.5 writes 27.5% less code than Opus 5 for the same tasks and holds the same pass rate. We ran Anthropic's latest flagship model through the same Java benchmark and the same Sonar LLM evaluation framework we use for every model.

Opus 5 wrote 2.3 times more code than the model before it, and the story in that evaluation was largely about what that extra output costs in the review process. Opus 5.5 moves in the opposite direction: less code to read, and fewer findings to triage.

This is a concise release. Less code, fewer total findings, lower code smell and vulnerability density, and fewer of the most severe findings in all three categories. Bug density per line and concurrency findings move the other way, and commenting is much lighter than Opus 5.

What benchmark and methodology did we use  to evaluate Claude Opus 5.5?

Model: Claude Opus 5.5 High

Baseline: Claude Opus 5 Thinking

Language: Java

Benchmark: the same Java benchmark we use for every model in our LLM leaderboard, covering HumanEval, MBPP and ComplexCodeEval. The Opus 5.5 run records 4,444 tasks. The pass rate covers the 544 HumanEval and MBPP tasks with executable tests. ComplexCodeEval contributes to the code analysis but not to the pass rate.

Analyzer: SonarQube algorithmic code analysis. Density metrics are per 1,000 lines of code (kLOC); category breakdowns are per million lines (mLOC).

Two terms worth defining first:

• Cyclomatic complexity: counts independent paths through a function.
• Cognitive complexity: a SonarQube metric that weights nested and deeply branched logic more heavily, reflecting how hard the code is for a human to read.

Neither one tells you whether the code works. Both help indicate how much logic reviewers and testers will need to reason through.

The severity tables below list every tier the analyzer reports, so each set of tiers sums to its headline density. Category rows are each rounded to the nearest whole number per million lines, so a column can land a point or two off its density.

The rule mapping behind the category tables is identical for both models on vulnerabilities and code smells.

How does Claude Opus 5.5 compare to Opus 5 across key metrics?

What is Claude Opus 5.5's functional pass rate?

87.68% across the 544 test-backed tasks, against 88.6% for Opus 5. That is 0.92 points lower, so the two releases sit within one percentage point of each other.

For context, the four Opus releases before this one landed at 82.72%, 83.66%, 82.9% and 88.6%. Opus 5 was the step change. Opus 5.5 holds that level rather than extending it, and does so while writing considerably less code, which is the more interesting part of this evaluation.

Missing completions improved, from 0.83% to 0.68%. That is 30 tasks where no parsable code came back, down from 37.

Roughly one in eight solutions in the test-backed subset still fails its tests. That is broadly where Opus 5 sat, and it is still a ratio that needs something downstream to catch it.

Does Claude Opus 5.5 generate less code than Opus 5?

Yes, and this is the change that shapes the rest of the evaluation.

Opus 5.5 generated 664,890 lines of code across the benchmark. Opus 5 generated 916,813. That is 27.5% less code for the same benchmark.

It used 89,466 functions to do that, down from 122,650, a 27.1% reduction. It is fewer functions and less code overall, at roughly the same granularity.

Comment lines fell further than the code did. Comment-line density is 3.1%, down from 10.5%, which in absolute terms is 21,058 comment lines against 107,517. Opus 5 was unusually heavily annotated for this benchmark.

For anyone maintaining this code later, there is less inline context to work from than Opus 5 provided, across a smaller codebase.

How complex is Claude Opus 5.5-generated code?

Cyclomatic complexity is 237.70 per kLOC against Opus 5's 237.82. Flat.

Cognitive complexity is 138.05 per kLOC, up from 132.29, a 4% increase.

So each thousand lines branches about as much as Opus 5's did, and nests slightly more deeply. Given that the total line count fell 27.5%, a 4% rise in cognitive density means the absolute amount of nested logic in the output came down substantially even as the per-line figure edged up.

Complexity density is close to flat across both measures. The meaningful complexity change in this release is the volume it is spread across.

Bug density and severity in Claude Opus 5.5

Bug density is 644 per mLOC, up from 576. An 11.8% increase.

BLOCKER reliability findings fell 41%, from 41 to 24 per mLOC. BLOCKER is the tier most likely to cause harm in production, and it is the tier that improved most here

The other three tiers rose. HIGH went from 28 to 33, MEDIUM from 241 to 263, and LOW from 265 to 323. Most of the density increase sits in LOW, which accounts for half the total.

In absolute terms the picture is different, because there is 27.5% less code. Opus 5.5 produced 428 bugs across the benchmark against Opus 5's 528, so 19% fewer bugs in total despite the higher rate per line.

The category breakdown shows where the movement is:

Five categories improved. API contract violations roughly halved, from 57 to 29 per mLOC. Control flow mistakes fell 57%. Type safety fell 24%, resource leaks 14%, and pattern and regex findings 25%.

Concurrency and threading rose 44%, from 205 to 295 per mLOC, and it remains the largest bug category by a wide margin. Performance and structure findings rose 87%, from 38 to 71. Exception handling rose 28%.

Concurrency led the bug profile in Opus 5 and the reasons are inherent to the category. Threading bugs are hard to reproduce, they depend on the environment they run in, and they surface as intermittent failures rather than clean ones. They are easy to miss in code review, which is what makes testing and analysis the right place to catch them.

What security vulnerabilities does Claude Opus 5.5-generated code contain?

Vulnerability density is 229 per mLOC, down from 251. A 9% reduction.

BLOCKER security findings fell 53%, from 19 to 9 per mLOC. HIGH findings fell 13%, from 195 to 170. Those are the two tiers carrying most of the weight, and both came down.

MEDIUM rose from 5 to 15 and LOW from 32 to 35, on a much smaller base. HIGH remains the dominant tier at 74% of security density, against 78% for Opus 5.

In absolute terms Opus 5.5 produced 152 vulnerabilities across the benchmark, against Opus 5's 230. That is 34% fewer.

The category breakdown:

The findings columns are the actual counts behind each rate, and they add up exactly. The per mLOC columns are each rounded to a whole number, so they can sum a little above the total: Opus 5.5 generated 664,890 lines, which makes a single finding worth 1.5 per mLOC, so a category with one finding shows as 2 and the column comes to 231 rather than 229. Opus 5 wrote more code, so its rate per finding is smaller and its column happens to land exactly on 251.

Cryptography misconfiguration is unchanged at 104 per mLOC against 103, and remains the largest security category. It covers weak algorithms, insecure key sizes, and random number generators used in unsafe ways. SonarQube’s algorithmic  analysis surfaces all of it reliably, which makes it the first place to point a pipeline.

Insecure system resource handling, the second largest category, fell 20% from 82 to 66. Hard-coded credentials fell from 14 to 2. Certificate validation findings do not appear in the Opus 5.5 output at all.

Injection attack findings rose from 7 to 17 per mLOC, and path traversal findings appeared at 5 where Opus 5 had none. Both are small categories in absolute terms, and both are well covered by data flow analysis.

How maintainable is Claude Opus 5.5-generated code?

Code smell density is 15.58 per kLOC, down from 19.69. A 21% reduction, and the largest move among the four headline density measures.

BLOCKER maintainability findings fell 20% and HIGH fell 23%. LOW halved, from 10,629 to 5,249 per mLOC, and that is where most of the density reduction comes from. MEDIUM rose 30%.

Seven of the ten named categories came down. Collection and generics parameter type fell 33%, from 8,901 to 6,004 per mLOC, and that single category accounts for most of the overall smell reduction. Dead code fell 28%, naming and documentation 21%, regex and string formatting 15%, design and framework 11%, and assignment and scope visibility 3%.

Collection and generics is still the largest smell category by a wide margin. These are raw types where parameterized generics belong, and collection handling that sidesteps type safety. In Java they carry a real cost: they suppress compiler warnings, make refactoring harder, and can hide the kind of bug a properly typed implementation would have caught at compile time. A 33% reduction on the dominant category is the single most useful maintainability result in this release.

The cognitive computational complexity smell rose 35%, from 790 to 1,065 per mLOC, which lines up with the cognitive complexity metric moving up 4%. Control and conditional logic and deprecated API usage each rose slightly.

How does Claude Opus 5.5 code volume affect total findings?

Opus 5 was the model where density and absolute counts told opposite stories. Opus 5.5 is the cleaner case, because volume and most densities moved the same way.

Total findings fell 42%. Bugs fell 19%, vulnerabilities fell 34%, and code smells fell 43%.

Three of the four density measures moved in the same direction: code smell density down 21%, overall issue density down 20%, vulnerability density down 9%. Only bug density went up, by 12%.

So for a team sizing review effort, Opus 5.5 produces less work than Opus 5 on every count. There is 27.5% less code to read and 42% fewer findings to triage. The per-line bug rate is higher, which matters when you are comparing how carefully each line needs checking, but it does not translate into more bugs to fix, because there are fewer lines producing them.

That is the distinction worth keeping straight. Density answers how clean the code is per unit written. Absolute counts answer how much work exists. For this release the two point the same way on three measures out of four, which is unusual and makes the result easier to act on than the Opus 5 numbers were.

How many output tokens does Claude Opus 5.5 use compared to Opus 5?

Input tokens were effectively identical across both runs at about 2.07 million, which is what you would expect from the same benchmark.

Output tokens fell 40%, from 21.71 million for Opus 5 to 12.96 million for Opus 5.5. Set against 27.5% less code, that means Opus 5.5 spent fewer tokens per line of output as well as producing fewer lines.

One caveat on totals. Opus 5.5 reports 3.23 million reasoning tokens, and the Opus 5 run recorded none, which almost certainly means the field was not captured rather than that no reasoning happened. So the output token comparison is the sound one, and totals across the two runs are not directly comparable.

What does this mean for teams using Opus 5.5?

Opus 5.5 holds Opus 5's pass rate while producing substantially less of everything else. 27.5% less code, 40% fewer output tokens, 42% fewer total findings, and lower density on code smells, vulnerabilities and overall issues. Fewer BLOCKER findings in all three categories. For teams where review capacity is the constraint, that combination is the most directly useful result we have measured from this family.

Three places to point verification effort.

Concurrency is first, and it is the same answer as the last several models we have evaluated. At 295 per mLOC it is the largest bug category and it rose 44% from Opus 5. 

It helps to know what this category actually flags, because it is narrower than concurrency in general. Lazy initialization written as double-checked locking. A lock acquired but not released on every path out of a method, which is usually a missing finally block. Synchronizing on a field that gets reassigned later, so two threads end up holding different locks. A counter incremented on a volatile field, where the read and the write are two separate operations. Calls to wait or notify made without holding the lock on the object. Thread.sleep called while a lock is still held.

Those patterns turn up in ordinary code: singleton and cache setup, connection and worker pools, shared counters and metrics, producer and consumer queues, and retry or polling loops. If you are generating any of that, prioritize testing and analysis for threading issues alongside code review, because these are the findings least likely to surface in a read-through.

Cryptography configuration is second. At 104 per mLOC it leads the security profile and it did not move from Opus 5. Together with insecure system resource handling it accounts for 170 of the 229 per mLOC, so roughly three quarters of the security surface sits in two categories that automated analysis covers well.

The per-line bug rate is third. Bug density rose 12% while total bugs fell 19%. Both are true and they answer different questions. If your gate is tuned to findings per thousand lines rather than to totals, this release will read as a regression on that one measure, and it is worth deciding in advance which view your process uses.

Lighter commenting is the other change to plan around. Comment-line density fell from 10.5% to 3.1%. Opus 5 was unusually well annotated, so this is a return toward the normal range rather than a departure from it, but if your team had started leaning on that inline context it will be thinner here.

Three takeaways:

• This is a concise release. 27.5% less code and 40% fewer output tokens for the same tasks, at a pass rate within one point of Opus 5. Total findings fell 42%.
• The most severe findings came down in all three categories. Reliability blockers fell 41%, security blockers 53%, and maintainability blockers 20%. Code smell density fell 21% and vulnerability density fell 9%.
• Bug density per line and concurrency are where to look. Bug density rose 12% to 644 per mLOC and concurrency findings rose 44% to 295, even as the absolute number of bugs fell 19%.

Opus 5.5 reverses the volume expansion that defined Opus 5, and it does so without giving back the correctness gain. That does not remove the need for code verification. It makes the verification cheaper, because there is less code and there are fewer findings, and it concentrates what is left in a small number of categories that automated analysis handles well.

These figures come from a pre-release build that we tested ahead of launch. We will refresh them on the Sonar LLM Leaderboard once Opus 5.5 reaches general availability, alongside every other model we have measured.