siliconflow/nimble

siliconflow/nimble

Description: Local typed decisions, contrastive data curation, and model evaluation.

Language: Python

Stars: 0

Forks: 0

Open issues: 0

Created: 2026-09-20T08:00:42Z

Pushed: 2026-09-21T05:30:50Z

Default branch: main

Fork: yes

Parent repository: bespokelabsai/nimble

Archived: no

README:

Bespoke Nimble

Data, Model, Recipe for an open Jev

Model · [Capabilities](#capabilities) · [Quickstart](#quickstart) · [Methodology](#methodology) · [Documentation and development](#documentation-and-development) · [Citation](#citation)


Nimble takes some text and a schema, and makes typed decisions about the text. The schema is the list of questions to answer. Each question is either a choice from a list that you give or a true or false question. For each question, Nimble returns the answer it picked and the probability of each allowed answer.

Nimble makes each decision in one step and does not write out any reasoning first, so it is fast (blazing fast!). Nimble is inspired by the System One approach of TypeSafe's Jev. In this repository, we share our recipe for training such a model.

Note that we did not distill from Jev. The point of the repository is to show how to curate data, how to train, and to serve such a model, and encourage more research!

You can run Bespoke-Nimble-9B on a Mac with Apple Silicon or on a machine with an NVIDIA GPU.

Capabilities

We built Nimble in one day, so expect some rough edges. What Nimble can do comes from two sources: the first is the base model, Qwen3.5-9B, the second is our training data, which we curated for a few specific domains.

What you can build

| Task | You define | You get back | | --- | --- | --- | | Route a request | The destinations and when each one applies | The chosen destination and the probability of each destination | | Check a condition | A yes or no question and the evidence | True or false, and the probability of each | | Apply a policy | The rules and the allowed outcomes | A typed decision based on the text you supply | | Rate an outcome | Ordered levels, each with clear criteria | The chosen level and the probability of each level |

You supply a context, which is the text to judge, and a schema. The schema must be flat, which means that it has no nested fields. Each field is an enum or a boolean. An enum field has a fixed list of string choices, and a boolean field is true or false.

Each allowed answer has a code that is one token long. The scorer reads the model's logits for these codes. Logits are the raw scores that the model gives to each token. The scorer turns the logits into probabilities with the softmax function. Our Python code then builds the output from these probabilities, so there is no generated JSON to parse. If a field is an ordered rating scale, your application can use the probabilities to calculate an expected level.

On a Mac, ParallelScorer processes the shared context once and then scores all the fields in parallel. The CUDA scorer scores each field on its own, with the full prompt each time. Both scorers return the typed output. They also return the logits and the probabilities of the candidate answers. Each field is scored separately, so one field cannot see the answer to another field.

What you cannot build with the current release

• Nimble accepts only text. You cannot use it to judge other kinds of input,

e.g., images. This is true even though the base model includes a vision part.

• Nimble only picks from the answers you supply. It cannot write text of its

own, e.g., an explanation. It also cannot return nested JSON or a piece of text taken from the context. An enum field can have 1 to 26 string choices, and a boolean field has two.

• The probabilities are not a guarantee that an answer is correct. Nimble scales

them so that they add up to 1 across the answers you supplied. A probability of 0.9 does not mean that the answer is right 90% of the time. If it is possible that none of your answers fit, add an answer that means "no match". Test any probability threshold on your own data before you rely on it.

• Each prompt can have at most 2,048 tokens. This limit includes the schema and

the part of the prompt that names the field to score. Nimble rejects longer prompts. Fields cannot depend on each other, so your code must check that the answers to different fields are consistent.

We trained Bespoke-Nimble-9B on 2,676 examples that we curated. So it's performance will depend on this data and the domains it comes from. So don't expect a lot of generalization. But we do see that Nibmle is overall better than it's base model Qwen3.5-9B in new domains.

Quickstart

Clone the repository and move into its folder. Run all the commands below from this folder.

git clone https://github.com/bespokelabsai/nimble.git nimble
cd nimble

Use Python 3.12. To run the model on a Mac, you need Apple Silicon. Python must also run directly on macOS so that it can use Metal, which is Apple's interface to the GPU. To run the model on Linux, you need an NVIDIA GPU that supports BF16, a 16-bit number format.

Without quantization, the 9B weights alone take about 18 GB. Quantization means storing the weights with fewer bits to save memory. The model needs more memory than this while it runs. The merge step below runs on the CPU. It needs extra RAM, and it needs disk space for both the base weights and the merged weights. A Mac with 64 GB of memory has more free memory for this than a machine with 24 GB.

Download the model

Create a Python environment for preparing the model. On Linux, you can also use this environment to run the model on the GPU. If you do, install a build of PyTorch with CUDA support that works with your GPU driver.

python3.12 -m venv .cache/venvs/nimble
source .cache/venvs/nimble/bin/activate
python -m pip install torch==2.8.0 -r requirements/training.txt...