# SamplingClient

## _class_ [**tinker.SamplingClient**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L88)( _holder_, _sampling_session_id_, _shadow=False_, _retry_config=None_, _subprocess_sampling=None_)

Client for text generation and inference from trained or base models.

The SamplingClient lets you generate text tokens from either a base model or from weights you've saved using a TrainingClient. You typically get one by calling `service_client.create_sampling_client()` or `training_client.save_weights_and_get_sampling_client()`.

**Key methods:**

- sample() - generate text completions with customizable parameters
- compute_logprobs() - get log probabilities for prompt tokens

**Create method parameters:**

- `model_path`: Path to saved model weights (starts with 'tinker://')
- `base_model`: Name of base model to use for inference (e.g., 'Qwen/Qwen3-8B')
- `retry_config`: Configuration for retrying failed requests

```
sampling_client = service_client.create_sampling_client(base_model="Qwen/Qwen3-8B")
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt=prompt, sampling_params=params, num_samples=1)
result = future.result()
```

**Parameters:**

- [**holder**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L130) ( _[InternalClientHolder](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/internal_client_holder.py#L185)_)  
- [**sampling_session_id**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L132) ( _str_)  
- [**shadow**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L133) ( _bool_, default: `False`)  
- [**retry_config**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L134) ( _[RetryConfig](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/retry_handler.py#L39) | None_, default: `None`)  
- [**subprocess_sampling**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L135) ( _bool | None_, default: `None`)

### [**sample**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L298)( _prompt_, _num_samples_, _sampling_params_, _include_prompt_logprobs=False_, _topk_prompt_logprobs=0_)

Generate text completions from the model.

**Parameters:**

- [**prompt**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L300) ( _types. [ModelInput](https://tinker-docs.thinking-machines-lab/tinker/api-reference/types/modelinput/)_) – The input tokens as ModelInput  
- [**num_samples**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L301) ( _int_) – Number of independent samples to generate  
- [**sampling_params**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L302) ( _types. [SamplingParams](https://tinker-docs.thinking-machines-lab/tinker/api-reference/types/samplingparams/)_) – Parameters controlling generation (temperature, max_tokens, etc.)  
- [**include_prompt_logprobs**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L303) ( _bool_, default: `False`) – Whether to include log probabilities for prompt tokens  
- [**topk_prompt_logprobs**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L304) ( _int_, default: `0`) – Number of top token log probabilities to return per position

**Returns:** A `Future` containing the [`SampleResponse`](https://tinker-docs.thinking-machines-lab/tinker/api-reference/types/sampleresponse/) with generated text

```
prompt = types.ModelInput.from_ints(tokenizer.encode("The weather today is"))
params = types.SamplingParams(max_tokens=20, temperature=0.7)
future = sampling_client.sample(prompt=prompt, sampling_params=params, num_samples=1)
result = future.result()
for sample in result.samples:
    print(tokenizer.decode(sample.tokens))
```

_Async variant:_`sample_async()`

### [**compute_logprobs**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L385)( _prompt_)

Compute log probabilities for prompt tokens.

**Parameters:**

- [**prompt**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L385) ( _types. [ModelInput](https://tinker-docs.thinking-machines-lab/tinker/api-reference/types/modelinput/)_) – The input tokens as ModelInput

**Returns:** A `Future` containing a list of log probabilities for each token in the prompt. None values indicate tokens where log probabilities couldn't be computed.

```
prompt = types.ModelInput.from_ints(tokenizer.encode("Hello world"))
future = sampling_client.compute_logprobs(prompt)
logprobs = future.result()
for i, logprob in enumerate(logprobs):
    if logprob is not None:
        print(f"Token {i}: logprob = {logprob:.4f}")
```

_Async variant:_`compute_logprobs_async()`

### [**get_tokenizer**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L443)()

Get the tokenizer for the current model.

**Returns:** `PreTrainedTokenizer` compatible with the model

### [**get_base_model**](https://github.com/thinking-machines-lab/tinker/blob/main/src/tinker/lib/public_interfaces/sampling_client.py#L452)()

Get the base model name for the current sampling session.

**Returns:** _str_

_Async variant:_`get_base_model_async()`

## Referenced by

- [ServiceClient.create_sampling_client](https://tinker-docs.thinking-machines-lab/tinker/api-reference/serviceclient/#create_sampling_client)
- [tinker_cookbook.completers.TinkerTokenCompleter](https://tinker-docs.thinking-machines-lab/tinker/docs/api-reference/completers/tinkertokencompleter/)
- [tinker_cookbook.eval.SamplingClientEvaluator.__call__](https://tinker-docs.thinking-machines-lab/tinker/docs/api-reference/eval/samplingclientevaluator/#samplingclientevaluator-call)
- [TrainingClient.create_sampling_client](https://tinker-docs.thinking-machines-lab/tinker/api-reference/trainingclient/#create_sampling_client)
- [TrainingClient.save_weights_and_get_sampling_client](https://tinker-docs.thinking-machines-lab/tinker/api-reference/trainingclient/#save_weights_and_get_sampling_client)