hugpadhugpad

TechxGenus / Jamba-v0.1-9B

Text Generation9.3B9410Updated Apr 8, 2024
Hugging Face

Downloads

94

Likes

10

Last updated

Apr 8, 2024

License

apache-2.0

A dense version of Jamba-v0.1, which extracts the weights of the first expert. It no longer uses MoE. Please refer to this script for details. It can use single 3090/4090 for inference, and the usage method is exactly the same as Jamba-v0.1.

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • jamba
  • text-generation
  • mamba
  • moe
  • custom_code

Technical specifications

Parameters9.3B
Librarytransformers
LanguagesNot available
Licenseapache-2.0