Migrating from closed to open source models, Together

Migrating from closed to open source models, Together Webflow Analyze/Optimize tracking bridge -->

The Open Source AI Stack: models, inference, routers, harnesses & tools →

📈 GLM-5.3 vs. GLM 5.3 Flash on DeepSWE →

🚀 DeepSeek V4 Pro 0813 vs. GPT-5.6 Sol on DeepSWE →

⚡ On-demand B200s now available on Together GPU Clusters →

All blog posts

Inference

Published 9/16/2026

Migrating from closed to open source models, Together

A practical, five-stage playbook for moving from closed models to open source models: discover, evaluate, adapt, decide, and take it to production.

Authors

Kareem Abdol-Hamid, Zain Hasan, Sonny Khan

Table of contents

40+ Models Chosen for Production...40+ Models Chosen for Production...40+ Models Chosen for Production...

Migrations are the bane of any mature company. Systems are deeply integrated, you have key stakeholders across domains, and a small improvement can take months or years to integrate in traditional cases. Recently, the latest candidate for migration has been closed source to open source models (OSM). The promise of cost savings and distributed ownership of technology has always been compelling. Is it worth it though?

Luckily, closed to open source breaks the tradition of slow and painful migration, especially if you adopt a managed service. These services keep up with the pace of AI innovation while offloading much of the complexity from your business. Now the blast radius is much smaller, and the technical stack you need to migrate is less complex. Harnesses, gateways, and tooling are all built to adopt a variety of models. You can take full advantage of this for a quick and holistic migration from closed to open source. We'll walk you through a successful migration strategy we've seen with our customers. This can take weeks to months rather than months to years. In future, we'll also provide assets to help you track your progress along the way. This is an overview. We will provide deep dives for each section we feel could use more color, with workbooks, examples, and walkthroughs. Discover As you start to discover candidate models, the most important thing is to know what you're evaluating for. No single model is the best across the board. Different models have different strengths across domains, and the better you define the problems you want solved, the easier it becomes to shortlist models that will actually work for you. Start by defining your use-case and workload in plain terms: what tasks the model handles, what a good response looks like, and what your traffic actually consists of. With your use-case defined, pre-existing benchmarks become useful as a filtering tool. Identify the benchmarks most relevant to your work so that leaderboard numbers correlate with performance on your tasks. Good sources of benchmarks include The Open Frontier , Artificial Analysis , Epoch AI , Vals AI , Scale's leaderboards , Intelligence and Arena . Within these you might find the following benchmarks useful: Coding agents: FrontierCode , DeepSWE , Terminal Bench 4 Chat and general assistant work: LMArena , AA-Intelligence Index Agents and tool-use: τ-bench , Agent Arena Vision: Vision Arena

Frontier Open (eg. GLM-5.3, Kimi K3), Value Tier (eg. DeepSeek-V4 Flash, GLM-5.3 Flash), Small and edge-deployable (eg. Qwen 3.8 27B, Muse Glimmer) Remember, your job is to identify which of these are relevant for you to narrow down candidate models, we'll talk more about custom evals later. Typically you'll find that there are three tiers of models available to pick from, as shown above. For your top candidates, go beyond raw performance. Look at cost per task, how many tokens a model burns per attempt, how many steps it needs, end-to-end runtime, and how quickly you can verify a correct answer. Lots of new benchmarks are now reporting these per-task numbers, and we believe these better correlate with actual model cost and performance. A model that scores a few points lower but completes tasks in half the tokens and half the time may be the better fit. Finally, kick the tires. Take a handful of representative tasks into a playground and see if your shortlisted models perform as expected. Watch for failure modes and ask whether it's the model failing or the harness and context around it. It's also worth seeing what other companies running these models are saying about their experience. You should leave this step with two to three models that you are ready to put through serious evals. Evaluate Evaluating models will be the most involved part of this process and certainly needs a blog of its own, but at a high level you will be evaluating two things: accuracy and performance . Accuracy defines the capabilities of the model. This can include instruction following, summarization, function calling, vision performance etc. This is everything the model can do regardless of where it runs or how fast it is. You want to evaluate if it can solve your workload needs. Performance covers how well the model runs. This changes based on things like model size and architecture. For these tests we don't care how well the model is responding to requests. We're evaluating whether this model meets the cost and user-feel, or latency, expectations needed to adopt an OSM. For both accuracy and performance you'll need to have a consistent mindset: benchmarks are directionally useful, true tests are done with real data. The best way to evaluate a workload is not to look up or run generic benchmarks. Workloads have different traffic shapes, niche response requirements, and request structures. No benchmark can represent that except your own. Luckily, if you're already using closed source models, a simple replay of existing traffic is all the data you need to benchmark. No custom suites, no large data set. This is, and always has been, the best way to bench even before we were benchmarking modern frontier models (HPC, databases, load balancing, etc). Ensure you're still testing against the targets defined earlier though. If you don't have data to replay you're not blocked! We can still size according to general usage patterns like input and output size and expected cache hit rate. If closed source providers don't provide these metrics, gateways like LiteLLM can help gather this information. We'll discuss this technique further in a deep dive blog. After you've found your data: determine...