# `tml-renderers`

[`tml-renderers`](https://pypi.org/project/tml-renderers/) is Thinking Machines Lab's high-performance rendering package to ensure reliable sampling and post-training. It converts chat messages, tool calls, images, and audio into the token and media representation Inkling expects. It accepts native, typed `tml_renderers.chat` objects as well as the OpenAI-compatible message dictionaries used by Cookbook datasets.

## Installation

Install a compatible release of the SDK, Cookbook, and [`tml-renderers`](https://pypi.org/project/tml-renderers/):

```
pip install tinker tinker-cookbook tml-renderers

# Or install the Cookbook's Inkling dependencies
pip install "tinker-cookbook[inkling]"
```

### Requirements:

- **PyTorch >= 2.10**, installed before importing `tml_renderers` (libtorch is not bundled).
- **Supported platforms:** x86_64 and ARM (aarch64) Linux and macOS.
- **CPython 3.11+.**

## Native chat messages

Native messages make media pointers and TMLv0-specific content explicit:

```
from tinker_cookbook import model_info
from tinker_cookbook.renderers import get_renderer
from tinker_cookbook.tokenizer_utils import get_tokenizer
from tml_renderers import chat

model_name = "thinkingmachines/Inkling"
renderer = get_renderer(
    model_info.get_recommended_renderer_name(model_name),
    get_tokenizer(model_name),
)

message = chat.Message(
    content=chat.Text("What is 2 + 2?"),
    author=chat.Author(chat.AuthorKind.User),
)
prompt = renderer.build_generation_prompt([message])
```

Use native messages when you want typed `ImagePointer` or `AudioPointer` objects.

## OpenAI-compatible Cookbook messages

Use ordinary Cookbook messages when your dataset or application already follows the OpenAI-compatible role/content schema:

```
messages = [\
    {"role": "system", "content": "Answer concisely."},\
    {"role": "user", "content": "What is 2 + 2?"},\
]
prompt = renderer.build_generation_prompt(messages)
```

`build_generation_prompt(...)` returns the `tinker.ModelInput` you feed to Inkling.

To inspect or round-trip the message conversion directly:

```
from tml_renderers import chat

tml_messages = chat.OpenAIMessage.from_oss_messages(messages)
openai_messages = chat.OpenAIMessage.to_oss_messages(tml_messages)
```

This is compatibility with the **message format used by the Cookbook**.

## Feed the input to Inkling

`renderer.build_generation_prompt(messages)` produces a model-ready prompt containing the text, media, and tool context from the conversation. Pass it to a sampling client:

```
import tinker

client = await tinker.ServiceClient().create_sampling_client_async(
    base_model=model_name
)
response = await client.sample_async(
    prompt=prompt,
    num_samples=1,
    sampling_params=tinker.SamplingParams(
        max_tokens=128, stop=renderer.get_stop_sequences()
    ),
)
message, termination = renderer.parse_response(response.sequences[0].tokens)
```

The stop sequences and parser come from the same TMLv0 renderer, so structured output such as tool calls can be recovered as Cookbook messages. See [Thinking effort](https://tinker-docs.thinkingmachines.ai/cookbook/inkling/thinking-effort/) to control how much Inkling reasons.

## Tool calls

Declare tools with the standard Cookbook `ToolSpec`, then render the resulting prefix together with the user's message:

```
from tinker_cookbook.renderers import Message, ToolSpec

tool = ToolSpec(
    name="get_weather",
    description="Get weather for a city.",
    parameters={"type": "object", "properties": {"city": {"type": "string"}}},
)
messages = renderer.create_conversation_prefix_with_tools(
    [tool], system_prompt="Use tools when needed."
)
messages.append(Message(role="user", content="What is the weather in Tokyo?"))
prompt = renderer.build_generation_prompt(messages)
```

After sampling, `renderer.parse_response(...)` returns a Cookbook assistant message. If Inkling invoked a tool, inspect its `tool_calls`, execute the function in your application, append a `role="tool"` result, and sample again.

Append tool results using the OpenAI-compatible Cookbook format:

```
messages.append({
    "role": "tool",
    "tool_call_id": "call_weather_001",
    "name": "get_weather",
    "content": '{"temperature_c":28}',
})
```

## Training

The same renderer supports supervised examples. Use the same value you chose for generation; see [Thinking effort](https://tinker-docs.thinkingmachines.ai/cookbook/inkling/thinking-effort/).

```
model_input, weights = renderer.build_supervised_example(
    [\
        {"role": "user", "content": "What is 2 + 2?"},\
        {"role": "assistant", "content": "4"},\
    ],
    effort=0.9,
)
```

`build_generation_prompt(...)` uses `render_for_completion(...)`; supervised construction uses `render_for_sft(...)` and returns aligned model input and loss weights.

## `tml_renderers` API reference

The Cookbook renderer wraps the lower-level `tml_renderers` package. Use the Cookbook renderer for the common path; call `tml_renderers` directly when you want raw token spans, the streaming parser, or your own Tinker integration.

Create a renderer from the native tokenizer:

```
from tml_renderers import tokenizers, v0

tml_renderer = v0.Renderer(tokenizers.o200k_base_chat())
```

### `v0.Renderer`

All methods accept a `MessageList`, `list[Message]`, or `list[OpenAIMessage]`.

| Method | Purpose |
| --- | --- |
| `render_for_completion(messages)` | Render a generation prompt; returns `(spans, parser)`. |
| `render_for_completion_with_effort(messages, effort)` | Same, with a thinking-effort system message (`effort` in `[0, 1)`). |
| `render_for_sft(messages)` | Render supervised examples with per-token loss weights; returns `list[TrainingExample]`. |
| `stop()` | Stop token IDs for sampling. |

``` 
spans, parser = tml_renderer.render_for_completion(messages)
training_examples = tml_renderer.render_for_sft(messages)
```

### `v0.Parser`

`render_for_completion(...)` returns a `Parser` for turning sampled tokens back into messages.

| Method | Purpose |
| --- | --- |
| `parse_tokens(tokens)` | Parse sampled token IDs into `list[Message]`. |
| `parse(spans)` | Parse rendered token spans into `list[Message]`. |
| `flush()` | Return any buffered trailing message. |
| `parse_token(token)` / `parse_updates(tokens)` / `flush_updates()` | Streaming: emit incremental `ParseUpdate`s as tokens arrive. |

``` 
messages = parser.parse_tokens(sampled_token_ids)
```

### SFT with native messages and `ModelEndSampling`

`ModelEndSampling` is the model's end-of-turn signal. The parser treats it as the stop and drops it from returned messages, so do **not** add it to a generation prompt.

If you are already working directly with native `tml_renderers.chat` types, those pass through the Cookbook renderer too, and `build_supervised_example` terminates model turns with `ModelEndSampling` automatically (adding it yourself is a no-op). When calling `tml_renderers.v0.Renderer.render_for_sft(...)` directly, you own the exact message sequence, so include `ModelEndSampling` after model turns:

```
from tml_renderers import chat

tml_messages = [\
    chat.Message(\
        author=chat.Author(chat.AuthorKind.User),\
        content=chat.Text("What is 2+2?"),\
    ),\
    chat.Message(\
        author=chat.Author(chat.AuthorKind.Model),\
        content=chat.Text("4."),\
    ),\
    chat.Message(\
        author=chat.Author(chat.AuthorKind.Model),\
        content=chat.ModelEndSampling(),\
    ),\
]\
\
training_examples = tml_renderer.render_for_sft(tml_messages)
```

This is the intended boundary: datasets and environments produce ordinary chat-completion-shaped messages, and both the Cookbook renderer and `from_oss_messages(...)` add these end-of-turn boundaries automatically; only direct `render_for_sft(...)` callers add them explicitly.

### Message conversion

`chat.OpenAIMessage` bridges OpenAI-compatible dictionaries and native messages.

| Method | Purpose |
| --- | --- |
| `OpenAIMessage.from_oss_messages(dicts)` | OpenAI-compatible dicts → `list[OpenAIMessage]`. |
| `OpenAIMessage.to_oss_messages(messages)` | `OpenAIMessage`s → OpenAI-compatible dicts. |
| `OpenAIMessage.from_messages(parsed)` | Native `Message`s → `list[OpenAIMessage]`. |

### Tinker adapters

`tml_renderers.tinker` converts rendered output into Tinker SDK objects.

| Function | Purpose |
| --- | --- |
| `token_spans_to_tinker_model_input(spans)` | Rendered spans → `tinker.ModelInput`. |
| `training_example_to_tinker_model_input_and_weights(example)` | A `TrainingExample` → `(ModelInput, weights)`. |

``` 
from tml_renderers.tinker import token_spans_to_tinker_model_input

model_input = token_spans_to_tinker_model_input(spans)
```

## Current constraints

- Inkling generates complete assistant messages; it does not continue from a partially written assistant message.
- `tml-renderers` renders whole conversations rather than individual messages.
- Native `tml_renderers.chat` inputs support training on all assistant messages. Selective Cookbook training modes require Cookbook/OpenAI-compatible dictionaries.