# Under the Hood

This page explains some implementation details of Tinker, which are important for understanding how to speed up your code.

## Clock Cycles

In Tinker, after you call `ServiceClient.create_lora_training_client`, your training job gets assigned to a pool of machines that work together -- a _worker pool_ which are doing forward-backward operations repeatedly in lock-step. Each of these steps of the worker pool is called a _clock cycle_. In each clock cycle, we do forward-backward and an optimizer step operation, each of which may involve multiple LoRA models that are being trained by this pool. You can think of this pool as a single large training run that is time-shared between multiple different LoRA models, often from different users.

With multi-tenancy -- sharing the same worker pool between multiple models -- we can run the training system efficiently even if users are training with small batch sizes, or if they have other delays in their training loops that would otherwise leave the worker pool idle. Small batch sizes can often give better _sample efficiency_, so this setup lets us achieve both high compute efficiency and high sample efficiency.

The downside is that it can sometimes lead to worse _latency_: even if training with a small batch, you'll still see the same step time as a large batch. (Still, note that we'll only charge you for the compute you use.) Also, if your training loop is implemented naively, you might have to wait multiple clock cycles per batch, because you might miss a clock cycle between operations.

### Overlapping `forward_backward` and `optim_step` Requests

As mentioned in the [Async Patterns](https://tinker-docs.thinkingmachines.ai/tutorials/basics/async-patterns/) tutorial, you should submit your `forward_backward` and `optim_step` requests together before waiting for either of them. This way, they'll end up on the same clock cycle. If you write the code naively, you'll end up using _three_ clock cycles per training step. Here's a recap of the example from the [Async Patterns](https://tinker-docs.thinkingmachines.ai/tutorials/basics/async-patterns/) tutorial:

Here's the difference visually:

Cycle | NCycle N+1 | Cycle N+2
-----|--------|---------
Naive | fwd_bwd | waiting... | optim_step  
Pipelined | fwd_bwd + optim | — | —

**❌ Naive implementation (uses 3 clock cycles):**

```python
# Submit forward_backward, gets queued for clock cycle N
fwd_bwd_future = await client.forward_backward_async(batch, loss_fn)

# Wait for it to complete, and for client to receive the result
# Due to communication latency, this happens a little after cycle N+1 started
fwd_bwd_result = await fwd_bwd_future

# Submit optim_step, gets queued for clock cycle N+2
optim_future = await client.optim_step_async(adam_params)

# Wait for it to complete, and for client to receive the result
# This happens a little after cycle N+2 finishes
optim_result = await optim_future

# Total: forward_backward on cycle N, optim_step on cycle N+2
# This takes 3 clock cycles (plus the time we waited before cycle N started)
```

**✓ Better implementation (uses 1 clock cycle):**

```python
# Submit both requests immediately. They'll both be slotted into the same clock cycle N
fwd_bwd_future = await client.forward_backward_async(batch, loss_fn)
optim_future = await client.optim_step_async(adam_params)

# Now wait for results - both operations happen on cycle N
fwd_bwd_result = await fwd_bwd_future
optim_result = await optim_future

# Total: both operations on cycle N
# This takes 1 clock cycle
```

### Pipelining to Maximize Clock Cycle Efficiency

To maximize efficiency and avoid missing clock cycles, you should **pipeline your training loop**: submit the next batch before waiting for the current batch to complete. This ensures there's always a request queued when a new clock cycle starts.

Cycle 1 | Cycle 2 | Cycle 3 | Cycle 4
---------|---------|---------|--------
Non-pipelined | batch 1 | gap | batch 2 | gap  
Pipelined | batch 1 | batch 2 | batch 3 | batch 4

**Non-pipelined**: Submits a batch, waits for it to complete, then submits the next. This wastes clock cycles because there's a gap between when one batch finishes and the next is submitted — typically using 2 clock cycles per training step.

**Pipelined**: Submits the next batch _before_ waiting for the previous batch to complete. This uses exactly 1 clock cycle per step, achieving maximum efficiency.

For hands-on examples of pipelined training, see the [Your First SFT](https://tinker-docs.thinkingmachines.ai/tutorials/basics/first-sft/) and [SFT with Config](https://tinker-docs.thinkingmachines.ai/tutorials/cookbook-abstractions/sft-with-config/) tutorials. For concurrent sampling patterns, see [Async Patterns](https://tinker-docs.thinkingmachines.ai/tutorials/basics/async-patterns/).