Chat completions

The chat completions endpoint generates text from a list of messages. It follows the OpenAI chat completions format, so most OpenAI-compatible clients work by changing the base URL and API key.

Endpoint

POST https://api.hyperbolic.ai/inference/v1/chat/completions

Send your sk_live_ API key in the Authorization header as Bearer YOUR_API_KEY. The examples use glm-5.3-flash. For other model IDs, see Available models.

Basic request

Python
TypeScript
cURL

import requests

url = "https://api.hyperbolic.ai/inference/v1/chat/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY"
}
data = {
    "model": "glm-5.3-flash",
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in simple terms."}
    ],
    "max_tokens": 2048,
    "temperature": 0.7,
    "top_p": 0.9
}

response = requests.post(url, headers=headers, json=data)
response.raise_for_status()
print(response.json()["choices"][0]["message"]["content"])

const response = await fetch("https://api.hyperbolic.ai/inference/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: "Bearer YOUR_API_KEY",
  },
  body: JSON.stringify({
    model: "glm-5.3-flash",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "Explain quantum computing in simple terms." },
    ],
    max_tokens: 2048,
    temperature: 0.7,
    top_p: 0.9,
  }),
});

const json = await response.json();
console.log(json.choices[0].message.content);

curl -X POST "https://api.hyperbolic.ai/inference/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Explain quantum computing in simple terms."}
    ],
    "max_tokens": 2048,
    "temperature": 0.7,
    "top_p": 0.9
  }'

Request parameters

Parameter	Type	Required	Description
`model`	string	Yes	The model ID, such as `glm-5.3-flash`. See Available models
`messages`	array	Yes	The conversation so far, as objects with `role` and `content`
`max_tokens`	integer	No	Maximum number of tokens to generate. For `glm-5.3-flash`, the default is 2,048, which can cut off a reasoning model mid-thought. `qwen3.8-27b` has no default: without `max_tokens`, it generates until the model stops or the context window fills. Set it explicitly. See Context and output limits
`temperature`	number	No	Sampling temperature. Higher values give more varied output
`top_p`	number	No	Nucleus sampling threshold, from 0 to 1
`stream`	boolean	No	Stream the response as server-sent events. Defaults to `false`
`stop`	string or array	No	Sequences that end generation when the model produces them. See Reasoning models
`tools`	array	No	Functions the model can call. See Tool calling
`tool_choice`	string	No	Set to `"auto"` to let the model decide whether to call a tool
`response_format`	object	No	Ask for JSON output. See Structured output

Other OpenAI chat completion parameters are passed to the model, and support for them depends on the model.

Messages

Each message has a role and content:

system: instructions that set the assistant’s behavior. Put it first.
user: a message from the user.
assistant: an earlier reply from the model.

Each request is independent. To continue a conversation, send the full history with each request:

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"},
    {"role": "assistant", "content": "The capital of France is Paris."},
    {"role": "user", "content": "What is its population?"}
  ]
}

Every message you send counts as input tokens, so long histories cost more. Repeated prompt prefixes may be billed at the cheaper cached input rate. See How tokens are billed.

Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1760000000,
  "model": "glm-5.3-flash",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Quantum computing uses quantum mechanics...",
        "reasoning_content": "The user wants a simple explanation..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 1102,
    "total_tokens": 1127,
    "prompt_tokens_details": {
      "cached_tokens": 0
    },
    "completion_tokens_details": {
      "reasoning_tokens": 892
    }
  }
}

The usage object reports the tokens used by the request:

prompt_tokens: input tokens. When present, prompt_tokens_details.cached_tokens is the part served from the prompt cache.
completion_tokens: output tokens, including any thinking. The thinking count is in completion_tokens_details.reasoning_tokens for glm-5.3-flash, and in a top-level reasoning_tokens field for qwen3.8-27b.

The exact fields in usage can vary by model.

Reasoning models

Reasoning models think before they answer. glm-5.3-flash and qwen3.8-27b return their thinking in a reasoning_content field, next to content in the message and in each streamed delta. The answer in content doesn’t include the thinking. Other models may not return their thinking. If a model includes its thinking in content between <think> and </think> tags, strip that section if you only want the answer. Keep these in mind when you use a reasoning model:

Thinking counts against max_tokens. If the model uses up max_tokens while it’s still thinking, the response ends with finish_reason set to "length" and an empty answer. Raise max_tokens and try again. Thinking length varies by prompt, and even a short question can use more than 1,000 tokens of thinking. Always set max_tokens: with glm-5.3-flash, leaving it out means a limit of 2,048, and with qwen3.8-27b it means no limit until the context window fills.
Thinking is billed as output tokens. It’s included in completion_tokens. See Response for where each model reports how much of it was thinking.
Leave earlier thinking out of the conversation history. In later requests, send only each earlier reply’s answer as the assistant message. With glm-5.3-flash, thinking sent back in reasoning_content is counted as input tokens on every turn.
Stop sequences also apply to the thinking. With glm-5.3-flash, if a stop sequence appears while the model is thinking, the response ends there, before any answer. Choose stop sequences that won’t appear in the reasoning.

Streaming

Set stream to true to receive tokens as they’re generated. The response is a stream of server-sent events. Each data: line holds a chunk with the next piece of text in choices[0].delta.content, and the stream ends with data: [DONE]. With glm-5.3-flash, the thinking streams first in choices[0].delta.reasoning_content. To get token counts for a streamed request, set stream_options to {"include_usage": true}. A chunk at the end of the stream then carries the usage object.

Use streaming for long generations, such as long answers or reasoning models that think for a while. A non-streaming request sends nothing until the whole response is ready, and a request that takes too long before it starts responding can time out. See Time limits.

OpenAI SDK
Python
cURL

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.hyperbolic.ai/inference/v1"
)

stream = client.chat.completions.create(
    model="glm-5.3-flash",
    messages=[{"role": "user", "content": "Write a short story about a robot."}],
    max_tokens=2048,
    stream=True,
    stream_options={"include_usage": True}
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print(f"\n\nTokens used: {chunk.usage.total_tokens}")

import json
import requests

url = "https://api.hyperbolic.ai/inference/v1/chat/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY"
}
data = {
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Write a short story about a robot."}],
    "max_tokens": 2048,
    "stream": True
}

with requests.post(url, headers=headers, json=data, stream=True) as response:
    response.raise_for_status()
    for line in response.iter_lines(decode_unicode=True):
        if not line or not line.startswith("data: "):
            continue
        payload = line[len("data: "):]
        if payload == "[DONE]":
            break
        chunk = json.loads(payload)
        if chunk["choices"]:
            print(chunk["choices"][0]["delta"].get("content") or "", end="", flush=True)

curl -N -X POST "https://api.hyperbolic.ai/inference/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Write a short story about a robot."}],
    "max_tokens": 2048,
    "stream": true
  }'

Time limits

Each request has a time limit, including streamed requests:

Model	Time limit
`glm-5.3-flash`, `qwen3.8-27b`	600 seconds (10 minutes)
All other models	300 seconds (5 minutes)

Non-streaming requests also time out if the response doesn’t start within about 60 seconds. For long generations, set stream to true.

Tool calling

Pass tools to let the model call functions you define. When the model decides to call one, the response has finish_reason set to "tool_calls" and a tool_calls array in the message. Each call has an id, the function name, and its arguments as a JSON string. Your code runs the function, then sends the result back so the model can answer:

Add the assistant message from the response, with its tool_calls, to messages.
For each call, add a message with role set to "tool", tool_call_id set to the call’s id, and the result as a string in content.
Send the request again with the same tools. The model answers using the results.

Tool calling works with glm-5.3-flash, glm-5.3, and qwen3.8-27b. Support on other models depends on the model.

Python
cURL

import json
import requests

url = "https://api.hyperbolic.ai/inference/v1/chat/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY"
}
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"]
            }
        }
    }
]
data = {
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "What's the weather in Paris right now?"}],
    "tools": tools,
    "tool_choice": "auto",
    "max_tokens": 4096
}

response = requests.post(url, headers=headers, json=data)
response.raise_for_status()
message = response.json()["choices"][0]["message"]

# Send back the assistant turn, then one tool message per call
data["messages"].append({
    "role": "assistant",
    "content": message.get("content") or "",
    "tool_calls": message["tool_calls"]
})
for call in message["tool_calls"]:
    arguments = json.loads(call["function"]["arguments"])  # {'city': 'Paris'}
    result = {"city": arguments["city"], "temperature_c": 17, "conditions": "light rain"}  # your function's output
    data["messages"].append({
        "role": "tool",
        "tool_call_id": call["id"],
        "content": json.dumps(result)
    })

data.pop("tool_choice")
response = requests.post(url, headers=headers, json=data)
response.raise_for_status()
print(response.json()["choices"][0]["message"]["content"])
# It's currently 17°C in Paris with light rain...

curl -X POST "https://api.hyperbolic.ai/inference/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "What'\''s the weather in Paris right now?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }],
    "tool_choice": "auto",
    "max_tokens": 4096
  }'

Structured output

Set response_format to get JSON back in content:

{"type": "json_schema", "json_schema": {...}} returns JSON that follows the schema you provide.
{"type": "json_object"} returns valid JSON without a fixed schema. Describe the keys you want in your prompt.

Structured output works with glm-5.3-flash, glm-5.3, and qwen3.8-27b. Support on other models depends on the model.

Python
cURL

import json
import requests

url = "https://api.hyperbolic.ai/inference/v1/chat/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_API_KEY"
}
data = {
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Give me the largest city in Japan."}],
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "name": "city",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "country": {"type": "string"},
                    "population": {"type": "integer"}
                },
                "required": ["name", "country", "population"],
                "additionalProperties": False
            }
        }
    },
    "max_tokens": 4096
}

response = requests.post(url, headers=headers, json=data)
response.raise_for_status()
city = json.loads(response.json()["choices"][0]["message"]["content"])
print(city["name"], city["country"], city["population"])

curl -X POST "https://api.hyperbolic.ai/inference/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Give me the largest city in Japan."}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "city",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "country": {"type": "string"},
            "population": {"type": "integer"}
          },
          "required": ["name", "country", "population"],
          "additionalProperties": false
        }
      }
    },
    "max_tokens": 4096
  }'

List models

GET /v1/models returns the models you can call, in the OpenAI list format. It needs your API key, like other requests.

curl "https://api.hyperbolic.ai/inference/v1/models" \
  -H "Authorization: Bearer YOUR_API_KEY"

Each entry in data has the model id to pass in the model field, and owned_by, the model’s creator:

{
  "object": "list",
  "data": [
    {"id": "glm-5.3-flash", "object": "model", "created": 1760000000, "owned_by": "z-ai"}
  ]
}

OpenAI SDK compatibility

Point an OpenAI SDK at the Hyperbolic base URL and use your Hyperbolic API key:

Python
Node.js

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.hyperbolic.ai/inference/v1"
)

response = client.chat.completions.create(
    model="glm-5.3-flash",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"}
    ],
    max_tokens=2048
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.HYPERBOLIC_API_KEY,
  baseURL: "https://api.hyperbolic.ai/inference/v1",
});

const response = await client.chat.completions.create({
  model: "glm-5.3-flash",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Hello!" },
  ],
  max_tokens: 2048,
});

console.log(response.choices[0].message.content);

Overview

GPUs

Managed Kubernetes

Rental Operator

Inference

Account & Billing

API Reference

References

Endpoint

Basic request

Request parameters

Messages

Response

Reasoning models

Streaming

Time limits

Tool calling

Structured output

List models

OpenAI SDK compatibility

Next steps

Errors

Inference overview

​Endpoint

​Basic request

​Request parameters

​Messages

​Response

​Reasoning models

​Streaming

​Time limits

​Tool calling

​Structured output

​List models

​OpenAI SDK compatibility

​Next steps

Errors

Inference overview

Endpoint

Basic request

Request parameters

Messages

Response

Reasoning models

Streaming

Time limits

Tool calling

Structured output

List models

OpenAI SDK compatibility

Next steps