Gold Like Answers Benchmarks

Gold-Like Answers Reveal LLM Judge Bias in Coding Benchmarks

Skip to Main Menu

Skip to Main Content

Skip to Footer

Back to Blog

-->

Back to Blog

TL;DR

While benchmarking our coding agent, we noticed our LLM judge — the component in Maestro that selects the best output from parallel agent runs — was performing suspiciously well. Contamination was the obvious suspect — but when we reproduced the same bias on a clean dataset the model couldn’t have seen, we realized something else was going on: the judge had learned to favor solutions that look like gold answers — minimal, clean, focused — over outputs that actually work. In this blog we isolate and quantify the effect of this ‘gold-like’ preference on the benchmark data and present a strategy for mitigating its impact in order to maintain the integrity of coding benchmark results.

A new data distortion culprit

When LLMs perform suspiciously well on public benchmarks, contamination is the usual culprit. So when we noticed that our reducer – an LLM Judge which selects the best patch from parallel agent runs – was demonstrating a clear preference for gold answers on SWE-bench Verified, we naturally suspected the model was recalling the dataset. Our doubts were further confirmed when OpenAI announced , two months later, that they had reached the same conclusion: Too many recent frontier models were familiar with the SWE-bench Verified dataset. SWE-bench was proclaimed contaminated.

Back in our lab, we replicated the experiment on SWE-rebench , a newer, contamination-resistant dataset; yet the preference for gold answers remained. That was a big alarm bell.

In this case, the LLM Judge wasn’t memorizing specific solutions – but it was learning the traits of “gold-like” patches, such as minimality and clarity. While these flavoring are typically considered positive, here, they introduced a misalignment with our success criteria : The model prioritized surface-level traits over functional correctness, often favoring a “clean” wrong solution over a “messy” correct one. It turns out that LLM Judges were delivering on form over function.

As we investigated, we found these seemingly ‘positive’ stylistic preferences can be just as distorting as explicit contamination. To tackle this, we redesigned our reducer prompts with concrete prioritization rules. Over a few iterations, we managed to create detailed enough prompt guidelines to align the model’s judgment with our actual success criteria. We share that journey below with other builders interested in investigating sources of contamination and distortion in their benchmark results and building more reliable measures of coding agent performance.

What are reducers? A quick intro

Since LLM-based agents are non-deterministic, running the same query multiple times produces different outputs; — with enough runs, at least one is likely correct. The challenge is identifying which one. This is the gap between pass@k (how often at least one of k runs gets it right) and the score you actually realize — which depends entirely on your ability to pick the correct output.

In Maestro, AI21’s orchestration meta model framework for agentic tasks, the system is spinning up multiple agents to work on the problem in parallel, and selecting the best output. That last step is handled by a component we call a reducer — essentially a selector that retrieves all output candidates and hopefully, if we configure it properly, picks the best one.

Reducers can make this selection in a few different ways. For example, some generic guidelines could be:

When creating a reducer for a specific use case, it may be beneficial to get specific; for example, we could instruct the reducer to pay more attention to the available context, or different parts of the output, and rank the candidates according to different criteria.

Same goes for the SWE-bench reducer. We needed to get specific in our guidelines.

Building a reducer for SWE-bench agents

The SWE-bench challenge

SWE-bench is a popular benchmark for evaluating coding agents. The input is a Github issue and a Docker image of a repository, and the expected output is the Git patch that contains the changes needed to resolve the issue. The agent’s job is to explore the repository, understand the issue and generate a patch. As long as the tests pass, the patch is considered successful (regardless of its code quality).

A common approach for SWE-bench agents is to use a ReAct loop with a single tool: a docker terminal that allows the agent to interact with the repository through bash commands. We started by implementing this agent within Maestro to collect a dataset of output patches. We generated the patches with two different models (GPT-5 mini and MiniMax-M2.1), and ran each example multiple times to get pass@k scores:

These results show the potential of a good SWE-bench reducer: the scores increase as we increase k , indicating that running multiple times on each instance returns a mixture of both ‘good’ and ‘bad’ patches. If we had a good reducer, we could achieve a significant improvement by just running the same agent multiple times and relying on the reducer to select the best candidate.

Adapting reducer variants to SWE-bench

Given what we understand about the SWE-bench benchmark, how do we implement a reducer that best supports our agent’s ability to output correct patches? Here’s how we adapted the three types of reducers outlined above to the SWE-bench setting:

Random reducer: Since the agent often finishes without making any changes, it’s not uncommon to get an empty patch as an output. In SWE-bench, we don’t get extra credit for “abstain”, so any reducer should start by filtering out candidates with an empty patch. This step automatically improves the score of the “random” reducer over that of pass@1.

Majority-vote reducer: Since the output here is a Git patch, it’s not trivial to define “identical” outputs. “Exact match” would be too strict, so we defined our matching logic to ignore formatting, comments and additional tests code. On average, the resulting score is usually higher than the random reducer, yet may dip below the random reducer on the more difficult examples.

LLM Judge: This kind of reducer can take on different levels of complexity and cost, depending on what information we include in the context (e.g. only the patch, the full agent trajectory, the repository code, etc.). Since the...