Why Agentic Ai Tools And Ai Agent Platforms Need Small Language Models Slms

Arcee AI | Why Agentic AI Tools and AI Agent Platforms Need Small Language Models (SLMs)

Trinity Large Thinking: Available on OpenRouter.

Try now ↗

ENTERPRISE

Research

COMPANY

Get API

Blog / Why Agentic AI Tools and AI Agent Platforms Need Small Language Models (SLMs)

Why Agentic AI Tools and AI Agent Platforms Need Small Language Models (SLMs) Andrew Walko ,

Chris Smith ,

Mary MacCarthy ,

•

February 25, 2025

Why does the model or (models) used in your agentic AI workflows matter? In this article we explain why the choice of models is key for a successful agentic AI strategy–and why small language models (SLMs) are the right choice.

When we first show our agentic AI platform to businesses, the most common question we hear is, “ Why would I need agentic AI that uses your models? I’m accustomed to using Claude [or insert any other popular LLM] and I don’t think a smaller model would work as well.” The thing is, just like you, we also think the LLMs (large language models) on the market are pretty incredible. These models are great to chat with because they are designed to interact with you , a human being. Yet the whole point of agentic AI is to get computers and machines doing work with minimal human intervention. This is where Arcee AI’s small language models (SLMs) stand out from the rest. While other models are optimized for back-and-forth discussion, our models are built to excel within automated systems. They’re trained to follow instructions precisely, and they have a deep understanding of API data. Here’s just one example: our 72B general purpose model, Virtuoso Large, outperforms the largest LLMs (estimated to be over 1.3 trillion parameters) on instruction following benchmarks like IFEval. In other words, the SLMs that power our agentic AI platform, Arcee Orchestra, are masters at getting stuff done–and they’re the ideal technology to power agentic AI. Small Language Models (SLMs): The Logical Fit for Agentic AI

Historically, in the machine learning/AI industry, there has been a perception that the bigger a model is, the more capable it is. This led to companies generally seeking to use the largest models they could find. But in today’s world, where models have surpassed trillions of parameters, using the largest models often leads to out-of-control costs and scaling issues. Also, large models are very difficult to train for specialized tasks. These challenges become even more insurmountable when companies try to incorporate them into an agentic AI solution–especially for tasks that require highly-specialized knowledge. Here at Arcee AI, we’re the industry leaders in small language models (SLMs). We’ve pioneered some of the top techniques ( model merging , distillation , Spectrum ) for training small models that compete with, and often outperform, their large counterparts. With the rise of agentic AI, we knew that we were perfectly positioned to provide the ideal models to power this technology. So we built an end-to-end agentic AI platform, Arcee Orchestra , that combines our SLMs with intelligent model routing and orchestration–enabling you to use the right model for the right tasks. This approach reduces costs, minimizes latency, and maintains or even improves accuracy. Your Workflows are Diverse–and Your Models Should Be, Too

For most businesses, your teams sometimes do need larger models. But they certainly do not need the biggest, most expensive LLM for routine work like summarizing meeting transcripts or generating follow-up emails. By offering a variety of model sizes, and intelligently routing your tasks to the right model, Orchestra dramatically reduces your cost without sacrificing performance. Seamless Integration for Effective AI Agent Workflows We also realized that for any agentic AI system to be valuable, it needed to be able to integrate with existing systems. We built Arcee Orchestra to include over 200 pre-built integrations to common applications like Salesforce, Slack, Dropbox, MS Office, GSuite, and others.  This means that you don't have to be the integrator (which is often one of the most costly barriers to successful agentic AI deployments). You can use the integrations directly out-of-the-box without any custom code. The fully integrated nature of Arcee Orchestra makes it incredibly easy not only to get started, but also to maintain and evolve your workflows. One development challenge that many customers don’t consider until it’s too late is that when models change, the optimal way to integrate and interact with them also changes. An AI agent that performs perfectly today may not work as well tomorrow, if a new model comes out and is integrated into the agent. With Arcee Orchestra, this complexity is abstracted away, and you get to utilize SOTA models without having to worry about a new model not working in your solution.

Specially-Trained Models for Agentic AI Workflows Arcee Orchestra comes with six SLMs out-of-the-box: three general purpose models (72B Virtuoso Large, 32B Virtuoso Medium, 14B Virtuoso Small) a coding model (32B Coder) a vision-language model (8B Spotlight) a reasoning model (32B Maestro).

What all of these models have in common: we put them through a highly-specific training process to make them excel at automated workflows and agentic AI systems. Let’s take a closer look at some of what we considered when training these models specifically for agentic AI. When you have different providers for models and frameworks/platforms, there’s always an inherent risk that performance will suffer, since the two components were not built to work together. Also, different models require different prompting strategies, which means that–in order to get various models and platforms working well together–you need expertise in all of them. This is expensive in terms of labor and time-to-value. Large third-party models are trained to excel at satisfying the (human) end user, by being conversational and having the ability to answer any query, such as “Create a song from the perspective of a giraffe.” These types of capabilities are great for consumers… but they increase the size of the model, and add nothing to what companies actually need the model to do for their business workflows. The Arcee AI SLMs that power Orchestra provide the requisite expertise to enable the platform to interface smoothly with a diversity of models and platforms, and to be able to execute business tasks. Our SLMs are carefully fine-tuned to be...