princeton-nlp / Llama-3-8B-ProLong-512k-Base

ProLong (<uPr</uincet<uo</un <ulong</u-context language models) is a family of long-context models that are continued trained and supervised fine-tuned from Llama-3-8B, with a maximum context window of 512K tokens. Our main ProLong model is one of the best-performing long-cont…