OpenAI-Compatible API - Tinker Documentation

OpenAI API Compatible Inference (in beta)

OpenAI-compatible inference lets you interact with any model checkpoint in Tinker, using an endpoint compatible with the OpenAI Completions API. It’s designed to let you easily “poke at” your model while you're training it.

For inference within your training runs (e.g. RL), we recommend using Tinker’s standard sampling client (see the API Reference).

Currently, OpenAI-compatible inference is meant for testing and internal use with low internal traffic, rather than large, high-throughput, user-facing deployments. Latency and throughput may vary by model and may change without notice during the beta. If you need higher or more stable throughput, contact the Tinker team in our Discord for guidance on larger-scale setups.

Use Cases

OpenAI-compatible inference is designed for:

We will release production-grade inference soon and will update our users then.

Using OpenAI compatible inference from an OpenAI client

The new interface exposes an OpenAI-compatible HTTP API. You can use any OpenAI SDK or HTTP client that lets you override the base URL.

  1. Set the base URL of your OpenAI-compatible client to:

    https://tinker.thinkingmachines.dev/services/tinker-prod/oai/api/v1

  2. Use a Tinker sampler weight path as the model name. For example:

    tinker://0034d8c9-0a88-52a9-b2b7-bce7cb1e6fef:train:0/sampler_weights/000080

    Any valid Tinker sampler checkpoint path works here. You can keep training and sample from the same checkpoint simultaneously.

  3. Authenticate with your Tinker API key, by passing the same key used for Tinker as the API key to the OpenAI client.

Note: We support both /completions and /chat/completions endpoints. For most use cases, we recommend /chat/completions. Here's how to decide which one to use:

Code Example

from os import getenv
from openai import OpenAI

BASE_URL = "https://tinker.thinkingmachines.dev/services/tinker-prod/oai/api/v1"
MODEL_PATH = "tinker://0034d8c9-0a88-52a9-b2b7-bce7cb1e6fef:train:0/sampler_weights/000080"

api_key = getenv("TINKER_API_KEY")

client = OpenAI(
    base_url=BASE_URL,
    api_key=api_key,
)

response = client.completions.create(
    model=MODEL_PATH,
    prompt="The capital of France is",
    max_tokens=50,
    temperature=0.7,  # example value, not a recommended default
    top_p=0.9,  # example value, not a recommended default
)

print(f"{response.choices[0].text}")

Notes:

Separating reasoning from response content

For reasoning models that emit chain-of-thought alongside their final answer, the /chat/completions endpoint accepts a non-standard separate_reasoning flag. When set to true, the server parses out the reasoning portion and returns it on a dedicated reasoning_content field rather than inlining it into content.

response = client.chat.completions.create(
    model=MODEL_PATH,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    extra_body={"separate_reasoning": True},
)

message = response.choices[0].message
print("Reasoning:", message.reasoning_content)
print("Answer:", message.content)

Notes:

Controlling thinking effort

For models that support it, the /chat/completions endpoint accepts the standard OpenAI reasoning_effort parameter to bias how much the model thinks before answering. Set it to one of the OpenAI strings — "none", "minimal", "low", "medium", "high", "xhigh" — or pass a raw float in [0.0, 0.99] for finer control.

response = client.chat.completions.create(
    model=MODEL_PATH,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    reasoning_effort="high",
)

You can also pass a float via extra_body:

response = client.chat.completions.create(
    model=MODEL_PATH,
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
    extra_body={"reasoning_effort": 0.8},
)

Notes: