hugpadhugpad

answerdotai / ModernBERT-large

Other396M656K501Updated Jan 15, 2025
Hugging Face

Downloads

656K

Likes

501

Last updated

Jan 15, 2025

License

apache-2.0

ModernBERT is a modernized bidirectional encoder-only Transformer model (BERT-style) pre-trained on 2 trillion tokens of English and code data with a native context length of up to 8,192 tokens. ModernBERT leverages recent architectural improvements such as:

Tasks & library

  • fill mask
  • transformers

Tags

  • transformers
  • pytorch
  • onnx
  • modernbert
  • fill-mask
  • masked-lm

Technical specifications

Parameters396M
Librarytransformers
Languagesen
Licenseapache-2.0