hugpadhugpad

TechxGenus / starcoder2-15b-instruct

Text Generation16B977Updated Mar 5, 2024
Hugging Face

Downloads

97

Likes

7

Last updated

Mar 5, 2024

License

bigcode-openrail-m

We've fine-tuned starcoder2-15b with an additional 0.7 billion high-quality, code-related tokens for 3 epochs. We used DeepSpeed ZeRO 3 and Flash Attention 2 to accelerate the training process. It achieves 77.4 pass@1 on HumanEval-Python. This model operates using the Alpaca i…

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • starcoder2
  • text-generation
  • code

Technical specifications

Parameters16B
Librarytransformers
LanguagesNot available
Licensebigcode-openrail-m