aws-prototyping / MegaBeam-Mistral-7B-512k

This model, presented in Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing, is a Long-Context LLM that supports 524,288 tokens in its context. MegaBeam-Mistral-7B-512k was trained on Mistral-7B Instruct-v0.2, and can be…