aifeifei798 / DualBigLittle-MoE-Qwen3-0.6b

Text Generation891M1040Updated Sep 30, 2026

Downloads

104

Likes

0

Last updated

Sep 30, 2026

License

apache-2.0

Conventional autoregressive Large Language Models (LLMs) and standard Mixture-of-Experts (MoE) architectures are fundamentally constrained by Multitask Negative Transfer (the "Seesaw Dilemma") and the Physical VRAM Capacity Wall: 1. Cognitive Interference: Forcing orthogonal d…

Tasks & library

  • text generation
  • transformers

Tags

  • transformers
  • dualbig_moe
  • text-generation
  • dualbig-moe
  • qwen3
  • moe

Technical specifications

Parameters891M
Librarytransformers
LanguagesNot available
Licenseapache-2.0