DeepSeek has released the preview version of its long-anticipated V4 series, pushing its open-source lineup into million-token territory with two Mixture-of-Experts variants. DeepSeek unveils the V4 series with a million-token context, new Sparse Attention, and open weights, aiming for open-source SOTA performance.
The structural headline is a new attention scheme pairing token-level compression with DeepSeek Sparse Attention, which the team credits for cutting long-context compute and memory costs sharply enough to make million-token inputs the baseline across DeepSeek.
🚀 DeepSeek-V4 Preview is officially live & open-sourced! Welcome to the era of cost-effective 1M context length.
🔹 DeepSeek-V4-Pro: 1.6T total / 49B active params. Performance rivaling the world’s top closed-source models.