siliconflow/Open-Jev

siliconflow/Open-Jev

Language: Python

License: MIT

Stars: 0

Forks: 0

Open issues: 0

Created: 2026-09-22T08:01:51Z

Pushed: 2026-09-23T03:24:17Z

Default branch: main

Fork: yes

Parent repository: Zefan-Cai/Open-Jev

Archived: no

README:

Open-Jev

Open probability decisions with Qwen3.5-2B, Qwen3.5-9B and Qwen3.8-27B. Supply a context, questions and candidates; get typed probabilities directly, without autoregressive answer generation or parsing generated JSON.

Project website: zefan-cai.github.io/open-jev — demos, measured results and project background.

Hugging Face: Collection · 2B model · 9B model · Dataset

Benchmarks: [Results and scope](docs/benchmarks.md) · Website tables · Source code

Public releases: the dataset and completed 2B/9B checkpoints are available at the links above. The 2B/9B artifacts are LoRA adapters plus a trained scalar decision head and calibration temperature. They require the pinned upstream Qwen weights and the Open-Jev loader; they are not merged base models or ordinary text-generation checkpoints. The dedicated public JevBench evaluation is complete. A new 27B training iteration with more diverse training data is training on four H100 GPUs, with optimizer updates verified at step 616 on September 21, 2026 at 07:31 UTC. Final checkpoint evaluation remains pending; finite training loss does not establish a quality improvement.

Open-Jev is an independent implementation inspired by TypeSafe's Jev. It does not reproduce proprietary RLCD, private weights or training data, and does not claim TypeSafe's advertised speedups or parity with every community demo.

Current results: [methods and quality](docs/provider-comparison.md) · interactive comparison · published X thread · [community cases and benchmark plan](docs/community-research-20260921.md).

New external evaluation: the [audited JevBench public-subset report](docs/jevbench-public.md) compares the released 2B/9B baselines, Jev, GPT-5.6 Luna and GPT-6 Astra on all 231 available public tasks. It includes every decision, per-tier results, probability validation and timing diagnostics, with candidate-order and hardware/network limitations. See the dedicated website table.

Broader data iteration: [new community cases](docs/community-research-broadening-20260921.md), [the frozen 148,639-row training mixture](docs/community-hard-training-v2.md), and the Chinese project story document the sources, construction, controls and limitations. New model gains have not been measured.

V3 data prepared: [natural intent routing](docs/community-routing-v3.md), [executable SQL](docs/sql-semantics-v3.md) and [approval/CMS controls](docs/community-workflow-v3.md) add 129,288 decision rows, including 74,921 training rows. The frozen next-stage mixture combines those training rows with 21,928 earlier replay rows for 96,849 training rows. The data are staged and tokenizer-checked; the next stage is queued behind completion and evaluation of the current 27B run. The [training recipe](docs/community-hard-training-v3.md) and [internal evaluation guide](docs/internal-evaluation.md) explain held-out splits, the fixed 1,280-row / 840-group comparison panel and equal-group reporting. These are preparation and audit results; the released-model scores below are unchanged.

Inference latency

The [latency report](docs/inference-latency.md) measures the released 2B checkpoint with warmed in-process Predictor calls and real loopback HTTP requests. It records P50/P95, context and candidate counts, hardware, cache mode, all warmup/timed attempts, and cache-output parity. The website latency table and evidence video use those saved measurements; video playback duration is not inference time.

The same 11 saved workloads were measured against Jev-1.13.0 from the same client: 220 measured HTTPS requests and 33 warmups, with no request errors. For customer service, median local Open-Jev HTTP latency is 85.03 ms versus 295.26 ms for Jev HTTPS. At 1024 state tokens and 32 candidates, Open-Jev is slower: 1015.90 ms versus 301.37 ms. Hardware and network paths differ; this is observed deployment latency, not matched-hardware speedup. CUDA prefix caching exceeded the probability tolerance on 9/11 workloads; all selected decisions matched, and caching remains off by default. Full results, including unfavorable cases, are in the [public evidence](reports/inference-latency/public).

The [provider comparison](docs/provider-comparison.md) also evaluates OpenAI structured decisions on identical saved requests and maintains a separate all-domain quality suite. Latency does not establish equal task quality. For the same customer-service request, OpenAI Luna and Astra have P50 response times of 918.13 ms and 1938.39 ms, respectively, with their recorded reasoning settings and structured categorical outputs.

Install and run

Use Python 3.10 or newer. Core task contracts, data generators and CPU tests need only the source package. Model inference and training share the train extra; a suitable GPU and the upstream model weights are required for the published checkpoint workflow. The full GPU workflow targets Linux.

git clone https://github.com/Zefan-Cai/Open-Jev.git
cd Open-Jev
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
python -m unittest discover -s tests -v

# Add the pinned model runtime dependencies.
python -m pip install -e '.[train]'
hf download ZefanCai/Open-Jev-2B \
--revision 0c7aa498b1627be8da4acf34c863ff0ee0a92785 --local-dir models/Open-Jev-2B
python -m jev.server --checkpoint models/Open-Jev-2B/package/checkpoint \
--device cuda:0 --max-length 4096 --batch-size 1 --no-prefix-cache

The download is pinned to the published 2B revision. The 9B package uses the same layout at revision 47e966881e489511c0c7f5633a9e1960a676a551. The public...