# tinker_cookbook.rl.ProblemEnv

## _class_ [**tinker_cookbook.rl.ProblemEnv**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L26)( _[Env](https://tinker-docs.thinkingmachines.ai/cookbook/api-reference/rl/env/)_)

A single-turn Q&A environment that rewards correct answers and valid formatting.

```python
class MathEnv(ProblemEnv):
    def __init__(self, renderer, question, answer):
        super().__init__(renderer)
        self.question = question
        self.answer = answer
    def get_question(self):
        return self.question
    def check_answer(self, response):
        return self.answer in response
    def check_format(self, response):
        return response.strip() != ""
    def get_reference_answer(self):
        return self.answer
```

### [**get_question**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L73)()  
Return the question text for this problem.

**Returns:** _str_

_Abstract method._

### [**check_answer**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L78)( _sample_str_)
Return a reward (0.0 to 1.0) for the model's response.

**Parameters:**
- [**sample_str**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L78) ( _str_) – The decoded text of the model's response.

**Returns:** _bool_ – Whether the answer is correct.

_Abstract method._

### [**check_format**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L90)( _sample_str_)
Return a format compliance reward (0.0 to 1.0).

**Parameters:**
- [**sample_str**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L90) ( _str_) – The decoded text of the model's response.

**Returns:** _bool_ – Whether the response follows the expected format.

_Abstract method._

### [**get_reference_answer**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L102)()
Return the reference answer for logging purposes.

**Returns:** _str_

_Abstract method._

### [**initial_observation**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L106)()
Build the initial prompt from the conversation prefix and question.

**Returns:** _tuple[Observation, StopCondition]_

### [**step**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L113)( _action_, _extra_)
Score the model's response for correctness and format compliance.

**Parameters:**
- [**action**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L113) ( _Action_) – Token IDs of the model's response.
- [**extra**](https://github.com/thinking-machines-lab/tinker-cookbook/blob/main/tinker_cookbook/rl/problem_env.py#L113) ( _[ActionExtra](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/actionextra/) | None_) – Optional action metadata (unused).

**Returns:** _[StepResult](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/stepresult/)_

## Referenced by
- [tinker_cookbook.distillation.PromptOnlyEnv](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/distillation/promptonlyenv/)
- [tinker_cookbook.rl.ProblemGroupBuilder](https://tinker-docs.thinking-machines.ai/cookbook/api-reference/rl/problemgroupbuilder/)