hugpadhugpad

answerdotai / ModernBERT-base

Other150M3.2M1.2KUpdated Jan 15, 2025
Hugging Face

Downloads

3.2M

Likes

1.2K

Last updated

Jan 15, 2025

License

apache-2.0

ModernBERT is a modernized bidirectional encoder-only Transformer model (BERT-style) pre-trained on 2 trillion tokens of English and code data with a native context length of up to 8,192 tokens. ModernBERT leverages recent architectural improvements such as:

Tasks & library

  • fill mask
  • transformers

Tags

  • transformers
  • pytorch
  • onnx
  • modernbert
  • fill-mask
  • masked-lm

Technical specifications

Parameters150M
Librarytransformers
Languagesen
Licenseapache-2.0