# tinker_cookbook.rl.Env

## _class_ [**tinker_cookbook.rl.Env**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L195)( _ABC_)

Stateful environment that a single agent interacts with.

Each [`Env`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/env/) instance is **single-use**: create it, run one episode, then discard it. Environments are created by `EnvGroupBuilder.make_envs`.

Implementors must override `initial_observation` and `step`.

```python
class MyEnv(Env):
    def __init__(self, question: str, answer: str, renderer):
        self.question = question
        self.answer = answer
        self.renderer = renderer
    
    async def initial_observation(self):
        messages = [{"role": "user", "content": self.question}]
        model_input, _ = self.renderer.build_generation_prompt(messages)
        return model_input, self.renderer.get_stop_sequences()
    
    async def step(self, action, *, extra=None):
        response = self.renderer.tokenizer.decode(action)
        reward = 1.0 if self.answer in response else 0.0
        return StepResult(
            reward=reward,
            episode_done=True,
            next_observation=tinker.ModelInput.from_ints([]),
            next_stop_condition=[],
        )
```

### [**initial_observation**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L228)()

Return the starting observation and stop condition for this episode.

**Returns:** tuple[Observation, StopCondition] | InitialObservationOverflow: The initial observation (model input) and the stop condition for the first generation step. Environments that enforce a token budget may instead return [`InitialObservationOverflow`](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/initialobservationoverflow/) when the initial prompt already exceeds it, which ends the rollout immediately and gracefully (no sampling call is made).

_Abstract method._

### [**step**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L245)( _action_, _extra_)

Advance the environment by one step given the agent's action.

**Parameters:**

- [**action**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L245) ( _Action_) – Token IDs produced by the agent.
- [**extra**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/types.py#L245) ( _[ActionExtra](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/actionextra/) | None_) – Optional metadata about the action, such as the stop reason.

**Returns:** _[StepResult](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/stepresult/)_ – The reward, next observation, and whether the episode is done.

_Abstract method._

## Referenced by

- [tinker_cookbook.rl.EnvFromMessageEnv](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/envfrommessageenv/)
- [tinker_cookbook.rl.EnvGroupBuilder.compute_group_rewards](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/envgroupbuilder/#envgroupbuilder-compute_group_rewards)
- [tinker_cookbook.rl.EnvGroupBuilder.make_envs](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/envgroupbuilder/#envgroupbuilder-make_envs)
- [tinker_cookbook.rl.ProblemEnv](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/problemenv/)
- [tinker_cookbook.rl.ProblemGroupBuilder.compute_group_rewards](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/problemgroupbuilder/#problemgroupbuilder-compute_group_rewards)
- [tinker_cookbook.rl.ProblemGroupBuilder.make_envs](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/problemgroupbuilder/#problemgroupbuilder-make_envs)
- [tinker_cookbook.rl.Trajectory](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/trajectory/)