GLM-5.3 vs. GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

GLM-5.3 vs. GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing Webflow Analyze/Optimize tracking bridge -->

🚀 DeepSeek V4 Pro 0813 vs. GPT-5.6 Sol on DeepSWE →

🤝 Together AI & Y Combinator announce partnership to deliver the first dedicated YC GPU cluster →

⚡ On-demand B200s now available on Together GPU Clusters →

🚀 Now serving MiniMax-M3 for efficient inference →

All blog posts

Model Library

Published 8/21/2026

GLM-5.3 vs. GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

Sol wins the first try, GLM-5.3 wins the rest at half the price, and the cascade beats both: 85.9% at \$6.61 a task.

Authors

Zain Hasan, Shobhit Dixit

Table of contents

40+ Models Chosen for Production...40+ Models Chosen for Production...40+ Models Chosen for Production...

Key Takeaways

Don't pick one. Run GLM-5.3 first, escalate to GPT-5.6 Sol when the tests fail. That cascade solves 85.9% of DeepSWE tasks at \$6.61 each. Sol alone solves 72.7% at \$8.37. Thirteen points better, 21% cheaper. Sol wins the single shot, narrowly. 72.7% pass@1 against GLM-5.3's 69.0%, a 3.7 point gap that sits inside a couple of standard deviations. GLM-5.3 wins every retry after that. It ties Sol at pass@2 (81.1 vs. 81.0) and leads pass@4 (87.6% vs. 85.8%). The price gap is 2.1x. \$3.99 per rollout against \$8.37. Per \$100 spent, GLM-5.3 solves 17 tasks and Sol solves 9. Sol is faster and steadier: 19 minutes and 61 steps per rollout against GLM's 35 and 124, with 61 tasks solved four for four against 48. GLM-5.3's failures are cleaner. It breaks tests that already passed in 11% of its failures, against 20% for Sol. Gate Sol's diffs on regressions. The two diverge (0.43 per-task correlation) and cover 106 of 113 tasks between them, which is what makes the cascade work.

We ran GLM-5.3 (max) against GPT-5.6 Sol (max) on all 113 DeepSWE tasks, four trials each, from the published per-trial records: 904 rollouts in total, 452 per side. Sol is the precision flagship. GLM-5.3 is the open-weight challenger that closed the gap. Every figure below comes from this run, so it can differ from other public GLM-5.3 vs. GPT-5.6 Sol scorecards.

DeepSWE · Head to Head

GLM 5.3 vs GPT 5.6 Sol at a glance

Model Pass@1 Avg cost Solves / $100 Out tok Steps

glm-5.3 [max] 69.0% ± 2.7% $3.99 17 80k 124

gpt-5.6-sol [max] 72.7% ± 2.2% $8.37 9 60k 61

GPT-5.6 Sol still holds the single-shot crown on DeepSWE, a benchmark that tests a model's software engineering ability across many task types and programming languages. GLM-5.3 arrives less than four points behind it at half the price and pulls ahead the moment you allow more than one attempt. This is the closest the open tier has come to the frontier, and the question worth answering is what Sol's remaining premium actually buys.

The DeepSWE scoreboard: pass@1 and pass@k Single shot, Sol edges ahead: 72.7% pass@1 to GLM-5.3's 69.0% under DeepSWE's official scoring. Allow retries and the order flips. At two attempts GLM-5.3 (81.1%) already ties Sol (81.0%); at four, GLM-5.3's 87.6% pass@4 leads 85.8%. The open model has the wider reach, so in any best-of-k setting it is the more accurate choice, and its extra attempts cost half of Sol's.

Cost comparison: GLM-5.3 vs. GPT-5.6 Sol pricing At \$3.99 a rollout, GLM-5.3 is 2.1x cheaper than Sol (\$8.37), which in value terms is 17 solves per \$100 against Sol's 9. Sol buys that premium back on latency: an average 19 minutes and 61 steps against GLM's 35 minutes and 124 steps, with 60k output tokens to GLM's 80k. The tradeoff is unusually clean. Sol is the faster, terser worker; GLM-5.3 is the lower-cost one that takes the long route. If a human is waiting, Sol earns its premium on latency alone. If a budget or a batch queue is waiting, GLM-5.3 is the better buy.

Coverage vs. reliability: precision vs. reach Decompose pass@1 and pass@4 into coverage (tasks solved at least once across four tries) and reliability (tasks solved on all four), and the split is clean. Sol is the precision corner: 84.5% reliability and 61 tasks solved four for four, the marks of a model that lands what it touches. GLM-5.3 trades to the other axis: wider coverage at 87.6% against 85.8%, but lower reliability at 78.8% and fewer solid tasks, 48 against 61. That coverage edge is the same fact as its pass@4 lead. GLM-5.3 touches more of the benchmark than Sol and converts each touch a little less often per shot. Failure modes: how each model gets it wrong The failure profiles differ sharply, and the split favors the open model. Sol breaks the repository's existing test suite in 20% of its failures, the GPT-family regression signature. GLM-5.3 does so in 11%; when it misses, it usually misses forward, a near miss with the baseline intact, at a 61% near-miss rate against Sol's 54%. So the lower-cost model is also the safer one to accept without a heavy regression gate. Put a full regression run around Sol before you take its diff. GLM-5.3 needs that guardrail less.

Where each wins, by task type The domain map splits evenly, four each. Sol takes data modeling and serialization (92%), build and ops tooling (73%), concurrency and durability (72%), and protocol conformance (59%), which is the exact-contract, systems-heavy work. GLM-5.3 takes query and config languages (88%, the highest single cell on the board), language and runtime internals (83%), stateful reactivity (73%), and program analysis, a 64 to 64 tie it holds on volume. That is the structured, interpreter-style work. GLM-5.3's one clear hole is protocol conformance at 44%, 15 points behind Sol. Sol has no single weak domain; it is simply strong across the board. Task types here were classified by an LLM from each benchmark prompt.

GLM-5.3 vs. GPT-5.6 Sol by programming language GLM-5.3's standout is JavaScript at 90%, 15 points clear of Sol's 75 and the best JS number of any model on the board. It also takes Rust, 70 to 60. Sol answers with Python (74 to 66), Go (79 to 76), and TypeScript (66 to 61). The routing rule is simple: JavaScript and Rust to GLM-5.3, the rest to Sol, with GLM-5.3 a lower-cost and close second almost everywhere.

How different are GLM-5.3 and GPT-5.6 Sol? Different enough to route on. Per-task correlation is 0.43, real disagreement for two models this close on aggregate. They both solve 90 tasks; GLM-5.3 alone gets 9, Sol alone gets 7, and 7 defeat both. Their union covers 106 of 113...