HyperbolicLabs/hyperbolic-gradio

HyperbolicLabs/hyperbolic-gradio

Language: Python

Stars: 18

Forks: 1

Open issues: 0

Created: 2024-11-14T01:25:42Z

Pushed: 2024-11-29T23:24:57Z

Default branch: master

Fork: no

Archived: no

README:

hyperbolic-gradio

is a Python package that makes it very easy for developers to create machine learning apps that are powered by Hyperbolic AI's API.

Installation

You can install hyperbolic-gradio directly using pip:

pip install hyperbolic-gradio

That's it!

Basic Usage

Just like if you were to use the hyperbolic API, you should first save your Hyperbolic API key to this environment variable:

export HYPERBOLIC_API_KEY=

Then in a Python file, write:

import gradio as gr
import hyperbolic_gradio

gr.load(
name='meta-llama/Meta-Llama-3-70B-Instruct',
src=hyperbolic_gradio.registry,
).launch()

Run the Python file, and you should see a Gradio Interface connected to the model on Hyperbolic AI!

!ChatInterface

Customization

Once you can create a Gradio UI from an Hyperbolic API endpoint, you can customize it by setting your own input and output components, or any other arguments to gr.Interface. For example, the screenshot below was generated with:

import gradio as gr
import hyperbolic_gradio

gr.load(
name='meta-llama/Meta-Llama-3-70B-Instruct',
src=hyperbolic_gradio.registry,
title='Hyperbolic-Gradio Integration',
description="Chat with Meta-Llama-3-70B-Instruct model.",
examples=["Explain quantum gravity to a 5-year old.", "How many R are there in the word Strawberry?"]
).launch()

!ChatInterface with customizations

Composition

Or use your loaded Interface within larger Gradio Web UIs, e.g.

import gradio as gr
import hyperbolic_gradio

with gr.Blocks() as demo:
with gr.Tab("Meta-Llama-3-70B-Instruct"):
gr.load('meta-llama/Meta-Llama-3-70B-Instruct', src=hyperbolic_gradio.registry)
with gr.Tab("Llama-3.2-3B-Instruct"):
gr.load('meta-llama/Llama-3.2-3B-Instruct', src=hyperbolic_gradio.registry)

demo.launch()

Under the Hood

The hyperbolic-gradio Python library has two dependencies: hyperbolic and gradio. It defines a "registry" function hyperbolic_gradio.registry, which takes in a model name and returns a Gradio app.

Supported Models in Hyperbolic AI

All chat API models supported by Hyperbolic AI are compatible with this integration. For a comprehensive list of available models and their specifications, please refer to the Hyperbolic AI Models documentation.

-------

Note: if you are getting a 401 authentication error, then the Hyperbolic API Client is not able to get the API token from the environment variable. This happened to me as well, in which case save it in your Python session, like this:

import os

os.environ["HYPERBOLIC_API_KEY"] = ...