The field of Artificial Intelligence is constantly pushing boundaries. Today, we’re excited to introduce INTELLECT-2, a groundbreaking 32-billion-parameter language model. What makes it truly special? It’s the first large-scale model of its kind trained using globally decentralized reinforcement learning (RL).
Built upon the capable QwQ-32B model, INTELLECT-2 represents a significant shift in how large AI models can be trained. Instead of relying on massive, centralized data centers, its training leveraged a distributed network of permissionless compute resources. This pioneering approach showcases the power of collaborative, decentralized AI development. Let’s dive into this exciting release.

Table of contents
- Introducing INTELLECT-2: A Leap in Distributed AI Training
- What Makes INTELLECT-2 Unique? The Power of Decentralized Training
- Under the Hood: The Technology Behind INTELLECT-2
- Putting INTELLECT-2 to Work: Usage Guide
- How Does INTELLECT-2 Perform? Benchmark Results
- The Significance of INTELLECT-2 for AI Development
- Conclusion: The Future is Distributed
Introducing INTELLECT-2: A Leap in Distributed AI Training
INTELLECT-2 isn’t just another large language model (LLM). It’s a testament to a new paradigm in AI training. The core innovation lies in its training methodology: a reinforcement learning process conducted across a dynamic, global network of independent compute contributors.
This marks a departure from traditional methods requiring co-located GPUs and high-speed interconnects. The INTELLECT-2 project demonstrates that complex RL training for massive models is inherently asynchronous and well-suited for the decentralized compute landscape. This 32B parameter reasoning model learned its skills through collaboration on an unprecedented scale.
What Makes INTELLECT-2 Unique? The Power of Decentralized Training
Training large models like INTELLECT-2 typically demands immense, centralized computing clusters. The INTELLECT-2 project flips this script by harnessing globally distributed, permissionless GPU resources contributed by the community.
This decentralized approach offers several key advantages:
- Asynchronous Operation: Training and inference (generating data) happen concurrently across the network, minimizing idle time and eliminating communication bottlenecks often seen in centralized setups.
- Heterogeneous Hardware Support: Contributors can participate using various hardware configurations, generating data at their own pace without needing uniform high-end GPUs.
- Lower Barrier to Entry: The majority of compute needed is for inference, which can run effectively on consumer-grade GPUs, broadening participation.
- Efficient Verification: Mechanisms are in place to verify the integrity of computations from untrusted sources efficiently.
This method proves that cutting-edge AI models can be developed collaboratively, leveraging diverse and distributed resources effectively.
Under the Hood: The Technology Behind INTELLECT-2
Bringing INTELLECT-2 to life required building novel infrastructure and refining training techniques. Several key components were developed from scratch:
PRIME-RL: Framework for Asynchronous RL
At the heart of the training process is PRIME-RL, a new framework purpose-built for distributed, asynchronous reinforcement learning. Unlike traditional frameworks that often handle training and inference sequentially in the same process, PRIME-RL decouples these tasks.
Model training occurs on trusted nodes, while data generation (rollouts) happens independently on the decentralized network of contributors. This separation, managed via exchanging data files and model checkpoints, creates an efficient pipeline without needing complex central orchestrators. PRIME-RL implements the GRPO training algorithm along with essential auxiliary losses.
You can explore the framework here: prime-rl on GitHub
Verifiable Rewards & Training Recipe
INTELLECT-2 was trained to improve its reasoning abilities, specifically in mathematics and coding. The training used a curated dataset comprising tasks where correctness could be automatically verified through symbolic methods or unit tests.
- Dataset: The training relied on high-quality math problems (NuminaMath-1.5, Deepscaler) and coding challenges (SYNTHETIC-1). This dataset is open-sourced.
- GRPO & Stability: The training used the GRPO (Group Relative Policy Optimization) algorithm, similar to approaches in models like Deepseek-R1. Crucially, modifications were made to enhance training stability at the 32B scale. This included techniques like two-sided GRPO clipping and aggressive gradient norm clipping to prevent the loss spikes and instabilities often seen in large model training.
- Length Control: An interesting aspect was training the model to adhere to a “thinking budget” specified in the prompt, using length rewards alongside task rewards.
Infrastructure Components: Enabling Trustless Compute
To manage the distributed network and ensure reliable training, several other key pieces of infrastructure were crucial:
- SHARDCAST: Efficiently broadcasts updated model weights (checkpoints) across the decentralized network using a tree-topology, similar to a CDN, ensuring inference workers get new policies quickly.
- TOPLOC: A novel hashing scheme used for verifiable inference. It allows validators to efficiently check if the computations (reasoning traces) submitted by inference workers were performed correctly using the right model weights, even across non-deterministic GPU hardware.
- Protocol Testnet: The underlying system (Prime Intellect Protocol) coordinates the permissionless nodes, manages contributions, and ensures the smooth operation of the entire distributed training run.
For a deep dive into the infrastructure and training specifics, please refer to the INTELLECT-2 Technical Report.
Putting INTELLECT-2 to Work: Usage Guide
Getting started with INTELLECT-2 is straightforward, thanks to its foundation on a popular architecture.
- Architecture: INTELLECT-2 uses the qwen2 architecture.
- Compatibility: This makes it compatible with widely-used inference libraries like vllm and sglang.
Important Usage Note: Because INTELLECT-2 was trained with a length control objective, you’ll achieve the best results by explicitly guiding its “thinking time.” Append the following phrase to your instruction prompt:
Think for 10000 tokens before giving a response.
The model trained on this objective, but the technical report notes that it didn’t train long enough to fully master precise length control. Therefore, experts recommend using the maximum trained target length (10,000 tokens) for optimal performance. However, the model also saw target lengths of 2000, 4000, 6000, and 8000 during training, so you might experiment with these values, although differences may not be drastic.
How Does INTELLECT-2 Perform? Benchmark Results
INTELLECT-2 demonstrates strong reasoning capabilities, showing improvements over its base model, QwQ-32B, in key areas. Here’s a look at its performance across common reasoning benchmarks:

| Model | AIME24 | AIME25 | LiveCodeBench (v5) | GPQA-Diamond | IFEval |
| INTELLECT-2 | 78.8 | 64.9 | 67.8 | 66.8 | 81.5 |
| QwQ-32B | 76.6 | 64.8 | 66.1 | 66.3 | 83.4 |
| Qwen-R1-Distill-32B | 69.9 | 58.4 | 55.1 | 65.2 | 72.0 |
| Deepseek-R1 | 78.6 | 65.1 | 64.1 | 71.6 | 82.7 |
Key Takeaways:
- INTELLECT-2 shows noticeable gains in mathematical reasoning (AIME24, AIME25) and coding ability (LiveCodeBench) compared to the QwQ-32B base model it was trained from.
- There’s a slight decrease in performance on IFEval (Instruction Following Evaluation). The likely reason is that the training data focused narrowly on math and coding tasks, rather than including diverse instruction-following datasets.
Overall, the results validate the effectiveness of the decentralized RL training approach for enhancing specific reasoning skills in a large language model.
The Significance of INTELLECT-2 for AI Development
The release of INTELLECT-2 is more than just a new model; it’s a proof-of-concept with significant implications:
- Viability of Decentralized Training: It demonstrates that training large, sophisticated AI models doesn’t solely belong to hyperscale data centers. Decentralized, collaborative efforts are feasible and effective.
- Importance of Test-Time Compute: The approach aligns with the growing trend of “test-time compute scaling,” where models spend more time “thinking” (inference) to solve complex problems. Decentralized networks are well-suited for this inference-heavy workload.
- Open Source Ecosystem: By open-sourcing the model, the training framework (PRIME-RL), and the dataset, the Prime Intellect team is fostering further research and development in this area.
- Future Potential: This opens the door for training even larger models collaboratively and exploring diverse RL environments contributed by the community. The asymmetry where inference (easily distributable) demands more compute than training (harder to distribute) makes decentralized RL particularly scalable.
Conclusion: The Future is Distributed
INTELLECT-2 stands as the first 32-billion-parameter language model trained through a globally distributed reinforcement learning effort. It showcases enhanced reasoning in math and code, achieved via the novel PRIME-RL framework and contributions from a permissionless network.
This project highlights the potential of decentralized compute for tackling the immense resource requirements of modern AI. The open-sourcing of INTELLECT-2 and its associated tools invites the broader community to explore, build upon, and contribute to the future of distributed AI development.
We encourage you to explore the model, delve into the technical report, examine the PRIME-RL code, and consider how decentralized approaches might reshape the AI landscape. The journey of INTELLECT-2 demonstrates that collaboration, powered by distributed resources, can drive the frontier of artificial intelligence.
| Latest From Us
- Forget Towers: Verizon and AST SpaceMobile Are Launching Cellular Service From Space

- This $1,600 Graphics Card Can Now Run $30,000 AI Models, Thanks to Huawei

- The Global AI Safety Train Leaves the Station: Is the U.S. Already Too Late?

- The AI Breakthrough That Solves Sparse Data: Meet the Interpolating Neural Network

- The AI Advantage: Why Defenders Must Adopt Claude to Secure Digital Infrastructure


