nvidia/Nemotron-3.5-Rerank-8B-BF16

Nemotron-3.5-Rerank-8B-BF16 Overview

Description:

Nemotron-3.5-Rerank-8B-BF16 was developed by NVIDIA for multilingual and multimodal question-answering retrieval. It is a world-class reranker optimized to produce a logit representing the relevance of a document to a text query. Documents may contain text, page images, or both, including charts, tables, and infographics. The model can be integrated into classical or agentic vision RAG pipelines. For text retrieval, the model was evaluated across 34 languages: English, Arabic, Assamese, Bengali, Bulgarian, Chinese, Danish, Dutch, Finnish, French, German, Hindi, Hinglish, Indonesian, Italian, Japanese, Korean, Malay, Marathi, Nepali, Norwegian, Persian, Portuguese, Romanian, Russian, Spanish, Swahili, Swedish, Tamil, Telugu, Thai, Ukrainian, Urdu, and Vietnamese. For image retrieval, it was evaluated across 18 languages: Arabic, Bengali, Chinese, English, Persian, Finnish, French, German, Hindi, Indonesian, Japanese, Korean, Russian, Spanish, Swahili, Telugu, Thai, and Yoruba. The model may support additional languages beyond these evaluation sets.

This model is ready for commercial or non-commercial use.

License/Terms of Use:

Governing Download Terms: Use of the model is governed by the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1).

Additional Information: The Ministral-3-8B-Instruct-2512-BF16 base model is governed by the Apache License, Version 2.0.

Deployment Geography:

Global

Use Case:

The reranking model serves as a key component of a multimodal retrieval system, such as a vision RAG pipeline, where it helps improve overall retrieval accuracy. A multimodal retrieval system typically uses a multimodal embedding model (dense retrieval) to return a set of potentially relevant documents given an input query. The reranking model then evaluates these candidate documents in greater detail and reorders them based on their relevance to the query, producing a final ranking. By refining the initial retrieval results, the reranking stage can improve the precision of the retrieved documents and ultimately lead to more accurate downstream responses.

Release Date:

Hugging Face 10/20/2026 via https://huggingface.co/nvidia/Nemotron-3.5-Rerank-8B-BF16

Model Architecture:

Architecture Type: Transformer

Network Architecture: The released model contains a Pixtral-based vision encoder and a bidirectional Ministral3 language-model tower. Training proceeded in two stages. In the first stage, an 8B reranker checkpoint derived from the Ministral-3-8B-Instruct-2512-BF16 backbone was trained on a blend of text-only datasets. In the second stage, the 8B reranker model initialized from the first-stage checkpoint was trained on image and combined image-text datasets.

Number of Model Parameters: 8.38B

Input:

Input Type(s): Text, Image

Input Format(s): Text: String; Image: Red, Green, Blue (RGB)

Input Parameters: Text: One-Dimensional (1D), Image: Two-Dimensional (2D)

Other Properties Related to Input: Each input is one text query paired with one document. The document may be an image, text, or a combination of image and text. The default image longest_edge is 1,120 pixels and can be increased to 1,540. The maximum context length is 32,768 tokens.

Output:

Output Type(s): Floats

Output Format: List of Floats

Output Parameters: One-Dimensional (1D)

Other Properties Related to Output: Each output value corresponds to a raw logit. The higher the value, the more relevant a document is to a given query.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Installation

Create a Python 3.12 environment for the Transformers and Sentence Transformers examples:

uv venv --python 3.12 .venv-transformers
source .venv-transformers/bin/activate
uv pip install torch "transformers>=5.5.0, None:
if not torch.cuda.is_available():
raise RuntimeError("This bfloat16 example requires a CUDA GPU.")

processor = AutoProcessor.from_pretrained(MODEL_PATH)

model = AutoModelForSequenceClassification.from_pretrained(
MODEL_PATH,
trust_remote_code=True,
dtype=torch.bfloat16,
device_map={"": DEVICE},
attn_implementation="flash_attention_2",
).eval()

def rerank(
query: str,
documents: list[dict],
) -> torch.Tensor:
scores = []
for document in documents:
document_content = []
if document.get("text") is not None:
document_content.append({"type": "text", "text": document["text"]})
if document.get("image") is not None:
document_content.append({"type": "image", "image": document["image"]})

inputs = processor.apply_chat_template(
[
{
"role": "query",
"content": [{"type": "text", "text": query}],
},
{"role": "document", "content": document_content},
],
tokenize=True,
add_generation_prompt=False,
return_dict=True,
return_tensors="pt",
)
if inputs["input_ids"].shape[-1] > MAX_MODEL_LEN:
raise ValueError("Input exceeds 32768 tokens; shorten the text or resize the image.")
inputs = inputs.to(DEVICE)

if inputs.get("pixel_values") is not None:
inputs["pixel_values"] = inputs["pixel_values"].to(model.dtype)

with torch.inference_mode():
score = model(**inputs).logits.squeeze().float()
scores.append(score)

# Higher scores indicate greater predicted relevance.
return torch.stack(scores)

query = "What is NVIDIA NeMo Framework?"
image = load_image(IMAGE_URL)
nemo_text = "NeMo Framework offers comprehensive functionalities for developing both LLMs and Multimodal Models."
biology_text = "A biological foundation model designed to analyze and generate DNA, RNA, and protein sequences."

documents = [
{
"name": "nemo_image_text",
"text": nemo_text,
"image": image,
},
{
"name": "nemo_text",
"text": nemo_text,
"image": None,
},
{
"name": "nemo_image",
"text": None,
"image": image,
},
{
"name": "biology_text",
"text": biology_text,
"image": None,
},
]

scores = rerank(
query=query,
documents=documents,
)

ranking = torch.argsort(scores, descending=True)

for rank, document_index in enumerate(ranking.tolist(), start=1):
document =...