hugpadhugpad

KnutJaegersberg / megatron-GPT-2-345m-EvolInstruct

Text Generation380M994Updated Nov 18, 2023
Hugging Face

Downloads

99

Likes

4

Last updated

Nov 18, 2023

License

cc-by-nc-4.0

Fine tuned small megatron-GPT-2 on Evol-Instruct dataset of WizardLM. Megatron-GPT-2-345m is competetive with GPT-2 large in some benchmarks. Megatron-GPT-2-345m performance on Wikitext-103 is 19.31, GPT2-large (1.5B) is 17.48 and GPT2-medium (762M) is 22.05.

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • pytorch
  • gpt2
  • text-generation

Technical specifications

Parameters380M
Librarytransformers
LanguagesNot available
Licensecc-by-nc-4.0