DatasetWithTeacher - Tinker Documentation
tinker_cookbook.distillation.DatasetWithTeacher
class tinker_cookbook.distillation.DatasetWithTeacher()
Pairs a supervised dataset with its teacher model.
Each domain in multi-teacher distillation gets one of these. The teacher's highest-probability tokens are used as soft targets for training.
Fields:
- dataset_builder ( _SupervisedDatasetBuilder _)
- teacher_config ( _TeacherConfig _)
- weight ( float, default:
1.0)