# TrainingClient

## _class_ [**tinker.TrainingClient**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L60)( _holder_, _model_seq_id_, _model_id_)

Client for training ML models with forward/backward passes and optimization.

The TrainingClient corresponds to a fine-tuned model that you can train and sample from.
You typically get one by calling `service_client.create_lora_training_client()`.  
**Key methods:**

- forward_backward() - compute gradients for training
- optim_step() - update model parameters with Adam optimizer
- save_weights_and_get_sampling_client() - export trained model for inference

```python
training_client = service_client.create_lora_training_client(base_model="Qwen/Qwen3-8B")
fwdbwd_future = training_client.forward_backward(training_data, "cross_entropy")
optim_future = training_client.optim_step(types.AdamParams(learning_rate=1e-4))
fwdbwd_result = fwdbwd_future.result()  # Wait for gradients
optim_result = optim_future.result()    # Wait for parameter update
sampling_client = training_client.save_weights_and_get_sampling_client("my-model")
```

**Parameters:**

- [**holder**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L85) ( _[InternalClientHolder](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/internal_client_holder.py#L185)_) – Internal client managing HTTP connections and async operations
- [**model_seq_id**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L85) ( _int_)
- [**model_id**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L85) ( _types. [ModelID](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/types/model_id.py#L5)_) – Unique identifier for the model to train. Required for training operations.

### [**forward**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L180)( _data_, _loss_fn_, _loss_fn_config=None_)

Compute forward pass without gradients.

**Parameters:**

- [**data**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L182) ( _List[types. [Datum](https://tinker-docs.thinking-machines.ai/tinker/api-reference/types/datum/)_ ) – List of training data samples
- [**loss_fn**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L183) ( _types. [LossFnType](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/types/loss_fn_type.py#L5)_) – Loss function type (e.g., "cross_entropy")
- [**loss_fn_config**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L184) ( _Dict[str, float] | None_, default: `None`) – Optional configuration for the loss function

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing the forward pass outputs and loss

```python
data = [types.Datum(
    model_input=types.ModelInput.from_ints(tokenizer.encode("Hello")),
    loss_fn_inputs={"target_tokens": types.ModelInput.from_ints(tokenizer.encode("world"))}
)]
future = training_client.forward(data, "cross_entropy")
result = await future
print(f"Loss: {result.loss}")
```

_Async variant:_`forward_async()`

### [**forward_backward**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L259)( _data_, _loss_fn_, _loss_fn_config=None_)

Compute forward pass and backward pass to calculate gradients.

**Parameters:**

- [**data**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L261) ( _List[types. [Datum](https://tinker-docs.thinking-machines.ai/tinker/api-reference/types/datum/)_ ) – List of training data samples
- [**loss_fn**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L262) ( _types. [LossFnType](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/types/loss_fn_type.py#L5)_) – Loss function type (e.g., "cross_entropy")
- [**loss_fn_config**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L263) ( _Dict[str, float] | None_, default: `None`) – Optional configuration for the loss function

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing the forward/backward outputs, loss, and gradients

```python
data = [types.Datum(
    model_input=types.ModelInput.from_ints(tokenizer.encode("Hello")),
    loss_fn_inputs={"target_tokens": types.ModelInput.from_ints(tokenizer.encode("world"))}
)]

# Compute gradients
fwdbwd_future = training_client.forward_backward(data, "cross_entropy")

# Update parameters
optim_future = training_client.optim_step(
types.AdamParams(learning_rate=1e-4)
)

fwdbwd_result = await fwdbwd_future
print(f"Loss: {fwdbwd_result.loss}")
```

_Async variant:_`forward_backward_async()`

### [**forward_backward_custom**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L393)( _data_, _loss_fn_, _loss_type_input='logprobs'_)

Compute forward/backward with a custom loss function.

Allows you to define custom loss functions that operate on log probabilities.
The custom function receives logprobs and computes loss and gradients.

**Parameters:**

- [**data**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L395) ( _List[types. [Datum](https://tinker-docs.thinking-machines.ai/tinker/api-reference/types/datum/)_ ) – List of training data samples
- [**loss_fn**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L396) ( _[CustomLossFnV1](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L52)_) – Custom loss function that takes (data, logprobs) and returns (loss, metrics)
- [**loss_type_input**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L398) ( _Literal['logprobs']_, default: `'logprobs'`) – Input space for `loss_fn`. Currently the only supported value is "logprobs".

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing the forward/backward outputs with custom loss

```python
def custom_loss(data, logprobs_list):
    # Custom loss computation
    loss = torch.mean(torch.stack([torch.mean(lp) for lp in logprobs_list]))
    metrics = {"custom_metric": loss.item()}
    return loss, metrics

future = training_client.forward_backward_custom(data, custom_loss)
result = future.result()
print(f"Custom loss: {result.loss}")
print(f"Metrics: {result.metrics}")
```

_Async variant:_`forward_backward_custom_async()`

### [**optim_step**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L557)( _adam_params_)

Update model parameters using Adam optimizer.

The Adam optimizer used by tinker is identical to [torch.optim.AdamW](https://docs.pytorch.org/docs/stable/generated/torch.optim.AdamW.html).
Note that unlike PyTorch, Tinker's default weight decay value is 0.0 (no weight decay).

**Parameters:**

- [**adam_params**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L557) ( _types. [AdamParams](https://tinker-docs.thinking-machines.ai/tinker/api-reference/types/adamparams/)_ ) – Adam optimizer parameters (learning_rate, betas, eps, weight_decay)

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing optimizer step response

```python
# First compute gradients
fwdbwd_future = training_client.forward_backward(data, "cross_entropy")

# Then update parameters
optim_future = training_client.optim_step(
types.AdamParams(
    learning_rate=1e-4,
    weight_decay=0.01
)
)

# Wait for both to complete
fwdbwd_result = await fwdbwd_future
optim_result = await optim_future
```

_Async variant:_`optim_step_async()`

### [**save_state**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L625)( _name_, _ttl_seconds=None_, _overwrite=False_)

Save model weights to persistent storage.

**Parameters:**

- [**name**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L626) ( _str_) – Name for the saved checkpoint
- [**ttl_seconds**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L626) ( _int | None_, default: `None`) – Optional TTL in seconds for the checkpoint (None = never expires)
- [**overwrite**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L626) ( _bool_, default: `False`) – If True, overwrite any existing checkpoint with the same name

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing the save response with checkpoint path

```python
# Save after training
save_future = training_client.save_state("checkpoint-001")
result = await save_future
print(f"Saved to: {result.path}")
```

_Async variant:_`save_state_async()`

### [**load_state**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L721)( _path_, _weights_access_token=None_)

Load model weights from a saved checkpoint.

This loads only the model weights, not optimizer state (e.g., Adam momentum).
To also restore optimizer state, use load_state_with_optimizer.

**Parameters:**

- [**path**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L722) ( _str_) – Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")
- [**weights_access_token**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L722) ( _str | None_, default: `None`) – Optional access token for loading checkpoints under a different account.

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing the load response

```python
# Load checkpoint to continue training (weights only, optimizer resets)
load_future = training_client.load_state("tinker://run-id/weights/checkpoint-001")
await load_future
# Continue training from loaded state
```

_Async variant:_`load_state_async()`

### [**load_state_with_optimizer**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L754)( _path_, _weights_access_token=None_)

Load model weights and optimizer state from a checkpoint.

**Parameters:**

- [**path**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L755) ( _str_) – Tinker path to saved weights (e.g., "tinker://run-id/weights/checkpoint-001")
- [**weights_access_token**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L755) ( _str | None_, default: `None`) – Optional access token for loading checkpoints under a different account.

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing the load response

```python
# Resume training with optimizer state
load_future = training_client.load_state_with_optimizer(
    "tinker://run-id/weights/checkpoint-001"
)
await load_future
# Continue training with restored optimizer momentum
```

_Async variant:_`load_state_with_optimizer_async()`

### [**save_weights_for_sampler**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L840)( _name_, _ttl_seconds=None_)

Save model weights for use with a SamplingClient.

**Parameters:**

- [**name**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L841) ( _str_) – Name for the saved sampler weights
- [**ttl_seconds**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L841) ( _int | None_, default: `None`) – Optional TTL in seconds for the checkpoint (None = never expires)

**Returns:** [`APIFuture`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/apifuture/) containing the save response with sampler path

```python
# Save weights for inference
save_future = training_client.save_weights_for_sampler("sampler-001")
result = await save_future
print(f"Sampler weights saved to: {result.path}")

# Use the path to create a sampling client
sampling_client = service_client.create_sampling_client(
    model_path=result.path
)
```

_Async variant:_`save_weights_for_sampler_async()`

### [**get_info**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L894)()

Get information about the current model.

**Returns:** [`GetInfoResponse`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/types/getinforesponse/) with model configuration and metadata

```python
info = training_client.get_info()
print(f"Model ID: {info.model_data.model_id}")
print(f"Base model: {info.model_data.model_name}")
print(f"LoRA rank: {info.model_data.lora_rank}")
```

_Async variant:_`get_info_async()`

### [**get_tokenizer**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L914)()

Get the tokenizer for the current model.

**Returns:**`PreTrainedTokenizer` compatible with the model

```python
tokenizer = training_client.get_tokenizer()
tokens = tokenizer.encode("Hello world")
text = tokenizer.decode(tokens)
```

### [**create_sampling_client**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L929)( _model_path_, _retry_config=None_)

Create a SamplingClient from saved weights.

**Parameters:**

- [**model_path**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L930) ( _str_) – Tinker path to saved weights
- [**retry_config**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L930) ( _[RetryConfig](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/retry_handler.py#L39) | None_, default: `None`) – Optional configuration for retrying failed requests

**Returns:** [`SamplingClient`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/samplingclient/) configured with the specified weights

```python
sampling_client = training_client.create_sampling_client(
    "tinker://run-id/weights/checkpoint-001"
)
# Use sampling_client for inference
```

_Async variant:_`create_sampling_client_async()`

### [**save_weights_and_get_sampling_client**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L961)( _name=None_, _retry_config=None_)

Save current weights and create a SamplingClient for inference.

**Parameters:**

- [**name**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L962) ( _str | None_, default: `None`) – Deprecated, has no effect. Will be removed in a future release.
- [**retry_config**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/training_client.py#L962) ( _[RetryConfig](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/retry_handler.py#L39) | None_, default: `None`) – Optional configuration for retrying failed requests

**Returns:** [`SamplingClient`](https://tinker-docs.thinking-machines.ai/tinker/api-reference/samplingclient/) configured with the current model weights

```python
# After training, create a sampling client directly
sampling_client = training_client.save_weights_and_get_sampling_client()

# Now use it for inference
prompt = types.ModelInput.from_ints(tokenizer.encode("Hello"))
params = types.SamplingParams(max_tokens=20)
result = sampling_client.sample(prompt, 1, params).result()
```

_Async variant:_`save_weights_and_get_sampling_client_async()`

## Referenced by

- [ServiceClient.create_lora_training_client](https://tinker-docs.thinking-machines.ai/tinker/api-reference/serviceclient/#create_lora_training_client)
- [ServiceClient.create_training_client_from_state](https://tinker-docs.thinking-machines.ai/tinker/api-reference/serviceclient/#create_training_client_from_state)
- [ServiceClient.create_training_client_from_state_with_optimizer](https://tinker-docs.thinking-machines.ai/tinker/api-reference/serviceclient/#create_training_client_from_state_with_optimizer)
- [tinker_cookbook.checkpoint_utils.save_checkpoint](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/checkpoint_utils/save_checkpoint/)
- [tinker_cookbook.eval.TrainingClientEvaluator.__call__](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/eval/trainingclientevaluator/#trainingclientevaluator-call)