# tinker_cookbook.rl

| Class | Description |
| --- | --- |
| [`StopReason`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/stopreason/) | Why a trajectory ended. |
| [`InitialObservationOverflow`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/initialobservationoverflow/) | Sentinel returned by `Env.initial_observation` instead of an |
| [`StepResult`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/stepresult/) | Result returned by `Env.step`. |
| [`Transition`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/transition/) | A single (observation, action, reward) tuple from a trajectory. |
| [`ActionExtra`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/actionextra/) | Extra metadata passed alongside an action to `Env.step`. |
| [`Env`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/env/) | Stateful environment that a single agent interacts with. |
| [`Trajectory`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/trajectory/) | A complete episode: a sequence of transitions from one agent in one environment. |
| [`RolloutError`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rollouterror/) | A captured error from a failed trajectory rollout. |
| [`EnvGroupBuilder`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/envgroupbuilder/) | Builds a group of environments. The group will be used in the following way: |
| [`TrajectoryGroup`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/trajectorygroup/) | A group of trajectories produced by one `EnvGroupBuilder`. |
| [`RLDataset`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rldataset/) | A dataset that produces batches of `EnvGroupBuilder` instances. |
| [`RLDatasetBuilder`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rldatasetbuilder/) | Abstract builder that constructs training and optional test RL datasets. |
| [`ProblemEnv`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemenv/) | A single-turn Q&A environment that rewards correct answers and valid formatting. |
| [`ProblemGroupBuilder`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/problemgroupbuilder/) | Builds a group of ProblemEnv instances from a factory callable. |
| [`MessageStepResult`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/messagestepresult/) | Result of a message-level step. |
| [`MessageEnv`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/messageenv/) | Abstract base class for message-level environments. |
| [`EnvFromMessageEnv`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/envfrommessageenv/) | Adapter that wraps a MessageEnv to implement the token-level Env interface. |
| [`RolloutStrategy`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/rolloutstrategy/) | Controls how trajectories are collected from a group of environments. |
| [`FailFast`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/failfast/) | Default strategy: any trajectory error crashes the group. |
| [`RetryOnFailure`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/retryonfailure/) | Retry failed or timed-out trajectories with fresh environments. |

| Function | Description |
| --- | --- |
| [`compute_advantages()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/compute_advantages/) | Compute advantages for each trajectory, centered within groups. |
| [`trajectory_to_data()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/trajectory_to_data/) | Return one or more Datum objects corresponding to the trajectory. |
| [`assemble_training_data()`](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/assemble_training_data/) | Convert trajectories to training data format.