Cerebras/cerebras-cloud-sdk-node

Cerebras/cerebras-cloud-sdk-node

Language: TypeScript

License: Apache-2.0

Stars: 76

Forks: 12

Open issues: 2

Created: 2024-07-31T18:27:00Z

Pushed: 2026-03-19T23:30:14Z

Default branch: main

Fork: no

Archived: no

README:

Cerebras Node API Library

This library provides convenient access to the Cerebras REST API from server-side TypeScript or JavaScript.

The REST API documentation can be found on inference-docs.cerebras.ai. The full API of this library can be found in [api.md](api.md).

It is generated with Stainless.

> [!NOTE] > This SDK has a mechanism that sends a few requests to /v1/tcp_warming upon construction to reduce the TTFT. If this behaviour is not desired, set warmTCPConnection=false in the constructor. > > If you are repeatedly reconstructing the SDK instance it will lead to poor performance. It is recommended that you construct the SDK once and reuse the instance if possible.

About Cerebras

At Cerebras, we've developed the world's largest and fastest AI processor, the Wafer-Scale Engine-3 (WSE-3). The Cerebras CS-3 system, powered by the WSE-3, represents a new class of AI supercomputer that sets the standard for generative AI training and inference with unparalleled performance and scalability.

With Cerebras as your inference provider, you can:

• Achieve unprecedented speed for AI inference workloads
• Build commercially with high throughput
• Effortlessly scale your AI workloads with our seamless clustering technology

Our CS-3 systems can be quickly and easily clustered to create the largest AI supercomputers in the world, making it simple to place and run the largest models. Leading corporations, research institutions, and governments are already using Cerebras solutions to develop proprietary models and train popular open-source models.

Want to experience the power of Cerebras? Check out our website for more resources and explore options for accessing our technology through the Cerebras Cloud or on-premise deployments!

Installation

npm install @cerebras/cerebras_cloud_sdk

API Key

Get an API Key from cloud.cerebras.ai and add it to your environment variables:

export CEREBRAS_API_KEY="your-api-key-here"

Usage

The full API of this library can be found in [api.md](api.md).

Chat Completion

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
apiKey: process.env['CEREBRAS_API_KEY'], // This is the default and can be omitted
});

async function main() {
const chatCompletion = await client.chat.completions.create({
model: 'llama3.1-8b',
messages: [{ role: 'user', content: 'Why is fast inference important?' }],
});

console.log(chatCompletion?.choices[0]?.message);
}

main();

Text Completion

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
apiKey: process.env['CEREBRAS_API_KEY'], // This is the default and can be omitted
});

async function main() {
const completion = await client.completions.create({
prompt: "It was a dark and stormy ",
model: 'llama3.1-8b',
});

console.log(completion?.choices[0]?.text);
}

main();

Streaming responses

We provide support for streaming responses using Server Sent Events (SSE).

Note that when streaming, usage and time_info will be information will only be included in the final chunk.

Chat Completion

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
apiKey: process.env['CEREBRAS_API_KEY'], // This is the default and can be omitted
});

async function main() {
const stream = await client.chat.completions.create({
messages: [{ role: 'user', content: 'Why is fast inference important?' }],
model: 'llama3.1-8b',
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
}

main();

Text Completion

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
apiKey: process.env['CEREBRAS_API_KEY'], // This is the default and can be omitted
});

async function main() {
const stream = await client.completions.create({
prompt: "It was a dark and stormy ",
model: 'llama3.1-8b',
max_tokens: 10,
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.text || '');
}
}

main();

If you need to cancel a stream, you can break from the loop or call stream.controller.abort().

Request & Response types

This library includes TypeScript definitions for all request params and response fields. You may import and use them like so:

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
apiKey: process.env['CEREBRAS_API_KEY'], // This is the default and can be omitted
});

const params: Cerebras.Chat.ChatCompletionCreateParams = {
model: 'llama3.1-8b',
messages: [{ role: 'user', content: 'Why is fast inference important?' }],
};
const chatCompletion: Cerebras.Chat.ChatCompletion = await client.chat.completions.create(params);

Documentation for each method, request param, and response field are available in docstrings and will appear on hover in most modern editors.

Handling errors

When the library is unable to connect to the API, or if the API returns a non-success status code (i.e., 4xx or 5xx response), a subclass of APIError will be thrown:

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
apiKey: process.env['CEREBRAS_API_KEY'], // This is the default and can be omitted
});

async function main() {
const chatCompletion = await client.chat.completions
.create({
model: 'some-model-that-doesnt-exist' as any,
messages: [{ role: 'user', content: 'This should cause an error!' }], // Ask TS to ignore the obviously invalid model name... Do not do this!
})
.catch(async (err) => {
if (err instanceof Cerebras.APIError) {
console.log(err.status); // 400
console.log(err.name); // BadRequestError
console.log(err.headers); // {server: 'nginx', ...}
console.log(err); // Full exception
} else {
throw err;
}
});
}

main();

Error codes are as follows:

| Status Code | Error Type | | ----------- | -------------------------- | | 400 | BadRequestError | | 401 | AuthenticationError | | 403 | PermissionDeniedError | | 404 | NotFoundError | | 422 | UnprocessableEntityError | | 429 | RateLimitError | |...