OpenAssistant / pythia-12b-pre-v8-12.5k-steps

This is an intermediate model used as base-model for further pythia 12b SFT-8 experiments. It was trained on a wider set of instruction-tuning datasets for 12.5k steps with batch-size 128 and a context size of 2048. The gpt4all dataset had "as a language model" contamination (…