hugpadhugpad

TechxGenus / starcoder2-3b-instruct

Text Generation3B1485Updated Mar 5, 2024
Hugging Face

Downloads

148

Likes

5

Last updated

Mar 5, 2024

License

bigcode-openrail-m

We've fine-tuned starcoder2-3b with an additional 0.7 billion high-quality, code-related tokens for 3 epochs. We used DeepSpeed ZeRO 3 and Flash Attention 2 to accelerate the training process. It achieves 65.9 pass@1 on HumanEval-Python. This model operates using the Alpaca in…

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • starcoder2
  • text-generation
  • code

Technical specifications

Parameters3B
Librarytransformers
LanguagesNot available
Licensebigcode-openrail-m