hugpadhugpad

allenai / longformer-base-4096

Other805.3K230Updated Apr 5, 2023
Hugging Face

Downloads

805.3K

Likes

230

Last updated

Apr 5, 2023

License

apache-2.0

longformer-base-4096 is a BERT-like model started from the RoBERTa checkpoint and pretrained for MLM on long documents. It supports sequences of length up to 4,096.

Tasks & library

  • transformers

Tags

  • transformers
  • pytorch
  • tf
  • rust
  • longformer
  • en

Technical specifications

ParametersNot available
Librarytransformers
Languagesen
Licenseapache-2.0