v2ray / dbrx-base-fixed
Text Generation132B406Updated Apr 10, 2024
Downloads
40
Likes
6
Last updated
Apr 10, 2024
License
databricks-open-model-license
The issues - How I fixed them: 1. Error when using gradient checkpointing - Fixed by using positional arguments instead because gradientcheckpointingfunc doesn't support kwargs. 2. VRAM usage go zoom and CUDA Out of Memory when backpropping through the MLP layer - Fixed by sep…
Tasks & library
- text generation
- transformers
Tags
- transformers
- dbrx
- text-generation
- conversational
- custom_code
Technical specifications
Parameters132B
Librarytransformers
LanguagesNot available
Licensedatabricks-open-model-license