tinker_cookbook.tokenizer_utils.TmlRenderersTokenizerAdapter

class tinker_cookbook.tokenizer_utils.TmlRenderersTokenizerAdapter()

Small tokenizer facade for tml_renderers models used through cookbook.

Fields:

• eos_token_id ( int | None, default: None)
• tml_tokenizer ( TmlTokenizer)