databricks/zerobus-sdk-java

databricks/zerobus-sdk-java

Description: Zerobus SDK for Java

Language: Java

License: NOASSERTION

Stars: 17

Forks: 0

Open issues: 0

Created: 2025-09-03T12:59:04Z

Pushed: 2026-03-06T14:25:05Z

Default branch: main

Fork: no

Archived: yes

README: > This repository has been moved to [databricks/zerobus-sdk](https://github.com/databricks/zerobus-sdk/tree/main/java). This repo is archived and read-only. All new development, issues, and pull requests should go to the monorepo. > > | SDK | Link | > |-----|------| > | Rust | databricks/zerobus-sdk/rust | > | TypeScript | databricks/zerobus-sdk/typescript | > | Java | databricks/zerobus-sdk/java | > | Python | databricks/zerobus-sdk/python | > | Go | databricks/zerobus-sdk/go |

Databricks Zerobus Ingest SDK for Java

GA: This SDK is generally available and supported for production use cases. Minor and patch version updates will not contain breaking changes. Major version updates may include breaking changes.

We are keen to hear feedback from you on this SDK. Please file issues, and we will address them.

The Databricks Zerobus Ingest SDK for Java provides a high-performance client for ingesting data directly into Databricks Delta tables using the Zerobus streaming protocol.

Table of Contents

• [Features](#features)
• [Architecture](#architecture)
• [Requirements](#requirements)
• [Quick Start User Guide](#quick-start-user-guide)
• [Prerequisites](#prerequisites)
• [Building Your Application](#building-your-application)
• [Choose Your Serialization Format](#choose-your-serialization-format)
• [Usage Examples](#usage-examples)
• [Protocol Buffers Examples](#protocol-buffers-examples)
• [JSON Examples](#json-examples)
• [API Styles](#api-styles)
• [Offset-Based API (Recommended)](#offset-based-api-recommended)
• [Future-Based API](#future-based-api)
• [Configuration](#configuration)
• [Logging](#logging)
• [Error Handling](#error-handling)
• [API Reference](#api-reference)
• [Best Practices](#best-practices)
• [Related Projects](#related-projects)
• [Changelog](#changelog)

Features

• High-throughput ingestion: Optimized for high-volume data ingestion via native Rust backend
• Native performance: JNI bindings to a high-performance Rust implementation
• Automatic recovery: Built-in retry and recovery mechanisms
• Flexible configuration: Customizable stream behavior and timeouts
• Protocol Buffers: Strongly-typed schema using protobuf
• JSON support: Ingest JSON records without Protocol Buffer schemas
• Offset-based API: Low-overhead alternative to CompletableFuture for high throughput
• OAuth 2.0 authentication: Secure authentication with client credentials

Architecture

The Java SDK uses JNI (Java Native Interface) to call a high-performance Rust implementation. This architecture provides:

• Lower latency: Direct native calls avoid Java gRPC overhead
• Reduced memory: Offset-based API eliminates CompletableFuture allocation per record
• Better throughput: Optimized Rust async runtime handles network I/O efficiently

┌───────────────────────────────────────────────────────────────┐
│ Java Application │
├───────────────────────────────────────────────────────────────┤
│ ZerobusSdk │ ZerobusProtoStream │ ZerobusJsonStream │
├───────────────────────────────────────────────────────────────┤
│ BaseZerobusStream (JNI) │
├───────────────────────────────────────────────────────────────┤
│ Native Rust SDK (libzerobus_jni) │
│ ┌─────────────┐ ┌─────────────┐ │
│ │ Tokio │ │ gRPC/ │ │
│ │ Runtime │ │ HTTP/2 │ │
│ └─────────────┘ └─────────────┘ │
└───────────────────────────────────────────────────────────────┘

Requirements

Runtime Requirements

• Java: 8 or higher - Download Java
• Databricks workspace with Zerobus access enabled

Supported Platforms

This SDK includes native libraries for the following platforms:

| Platform | Architecture | Status | |----------|--------------|--------| | Linux | x86_64 | Supported | | Windows | x86_64 | Supported | | macOS | x86_64 | Supported | | macOS | aarch64 (Apple Silicon) | Supported |

Dependencies

When using the fat JAR (recommended for most users):

• No additional dependencies required - all dependencies are bundled

When using the regular JAR:

• `protobuf-java` 4.33.0
• `slf4j-api` 2.0.17
• An SLF4J implementation such as `slf4j-simple` 2.0.17 or `logback-classic` 1.4.14

Build Requirements (only for building from source)

• Java: 8 or higher - Download Java
• Maven: 3.6 or higher - Download Maven
• Protocol Buffers Compiler (protoc): 33.0 - Download protoc (for compiling your own .proto schemas)

Quick Start User Guide

Prerequisites

Before using the SDK, you'll need the following:

1. Workspace URL and Workspace ID

After logging into your Databricks workspace, look at the browser URL:

https://.cloud.databricks.com/o=

• Workspace URL: The part before /o= → https://.cloud.databricks.com
• Workspace ID: The part after /o= → ``

Example:

• Full URL: https://dbc-a1b2c3d4-e5f6.cloud.databricks.com/o=1234567890123456
• Workspace URL: https://dbc-a1b2c3d4-e5f6.cloud.databricks.com
• Workspace ID: 1234567890123456

2. Create a Delta Table

Create a table using Databricks SQL:

CREATE TABLE .default.air_quality (
device_name STRING,
temp INT,
humidity BIGINT
)
USING DELTA;

Replace ` with your catalog name (e.g., main`).

3. Create a Service Principal

1. Navigate to Settings > Identity and Access in your...