tinker_cookbook.tokenizer_utils.SupportsTmlTokenizer

class tinker_cookbook.tokenizer_utils.SupportsTmlTokenizer( Protocol)

Tokenizer facade that exposes an underlying TmlTokenizer.

Fields:

• tml_tokenizer ( TmlTokenizer)