TechxGenus / starcoder2-7b-instruct
Text Generation7.2B1964Updated Mar 5, 2024
Downloads
196
Likes
4
Last updated
Mar 5, 2024
License
bigcode-openrail-m
We've fine-tuned starcoder2-7b with an additional 0.7 billion high-quality, code-related tokens for 3 epochs. We used DeepSpeed ZeRO 3 and Flash Attention 2 to accelerate the training process. It achieves 73.2 pass@1 on HumanEval-Python. This model operates using the Alpaca in…
Tasks & library
- text generation
- transformers
Tags
- transformers
- starcoder2
- text-generation
- code
Technical specifications
Parameters7.2B
Librarytransformers
LanguagesNot available
Licensebigcode-openrail-m