Quasar Models has introduced its new foundation model with 120 billion parameters, now pretrained on 7.5 trillion tokens. This release builds upon the success of their previous 20 billion parameter model, which improved 7 of 9 benchmarks and expanded the training context from 20,000 to 1 million tokens. After initial training on the SN24 and the 120-H200 cluster, the model will continue to undergo collaborative pretraining by miners on a dedicated subnet, aiming to reach a total of 15 trillion tokens.

Quasar: Quasar is the foundation model series created by QuasarModels and trained via decentralized networks. The project has returned its newest iteration to the subnet for continued pretraining by miners after early development stages. Previous versions demonstrated improvements across multiple benchmarks through internet-scale distributed training.
QuasarModels: QuasarModels develops and releases foundation models through decentralized pretraining processes involving subnets and distributed miners. The team recently announced advancing their latest model to a 120 billion parameter scale following initial pretraining on a large token dataset and cluster-based refinement. This work builds on their prior decentralized training runs that enhanced benchmark performance and extended context lengths.

Model Progression: This release follows an earlier decentralized training run that achieved benchmark gains and expanded context handling.
Decentralized Training: The model undergoes collaborative pretraining by miners on a dedicated subnet following initial cluster-based work.