hugpadhugpad

alignment-handbook / zephyr-7b-dpo-qlora

Other569Updated Jan 9, 2024
Hugging Face

Downloads

56

Likes

9

Last updated

Jan 9, 2024

License

apache-2.0

This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-qlora on the HuggingFaceH4/ultrafeedbackbinarized dataset. It achieves the following results on the evaluation set: - Loss: 0.5473 - Rewards/chosen: -0.8609 - Rewards/rejected: -1.5251 - Rewards/accuracies:…

Tasks & library

  • peft

Tags

  • peft
  • tensorboard
  • mistral
  • alignment-handbook
  • generated_from_trainer
  • trl

Technical specifications

ParametersNot available
Librarypeft
LanguagesNot available
Base modelmistralai/Mistral-7B-v0.1
Licenseapache-2.0