hugpadhugpad

allenai / biomed_roberta_base

Other28.7K31Updated Oct 3, 2022
Hugging Face

Downloads

28.7K

Likes

31

Last updated

Oct 3, 2022

License

Not available

BioMed-RoBERTa-base is a language model based on the RoBERTa-base (Liu et. al, 2019) architecture. We adapt RoBERTa-base to 2.68 million scientific papers from the Semantic Scholar corpus via continued pretraining. This amounts to 7.55B tokens and 47GB of data. We use the full…

Tasks & library

  • transformers

Tags

  • transformers
  • pytorch
  • jax
  • roberta
  • en

Technical specifications

ParametersNot available
Librarytransformers
Languagesen
LicenseNot available