DeepSeek V3-0324 Now the Top Non-Reasoning AI Model Even Surpassing Sonnet!
Ghufran Kazmi
DeepSeek V3-0324 Now the Top Non-Reasoning AI Model Even Surpassing Sonnet!
DeepSeek has just released a new checkpoint for its DeepSeek V3 model, labeled V3-0324, and the improvements are significant. This update brings enhancements in code creativity and reasoning abilities, putting it on par with some of the leading models in the field. Let’s dive into what makes this update so exciting.
DeepSeek V3-0324 Matches Sonnet 3.7 in Code Creativity
The ability of a Large Language Model (LLM) to generate creative and complex code is a crucial benchmark. A fascinating test has emerged to measure this: asking LLMs to write a raytracer in Python.
The Raytracer Challenge
The prompt is simple:
“Write a raytracer that renders an interesting scene with many colourful lightsources in python. Output a 800×600 image as a png”
This task tests the model’s ability to not just write code, but to do so in a way that shows creativity. Most LLMs, when given this prompt, produce a basic scene with simple red, green, and blue spheres. This suggests they rely on common examples found in their training data.
Summary of all tested LLMs
Sonnet’s Breakthrough
However, Anthropic’s Sonnet 3.5, and especially Sonnet 3.7, stood out. They generated more complex and varied scenes with aesthetically pleasing colors. This also resulted in larger file sizes, indicating more intricate code. Anthropic seemingly unlocked a way to boost code creativity, resulting in more visually appealing outputs.
DeepSeek V3-0324 Steps Up
The exciting news is that DeepSeek V3-0324 has now matched Sonnet 3.7 in this benchmark. This is a huge improvement over the previous V3 version. The model is now capable of producing similarly complex and creative raytracer outputs.
Weather App Creation: DeepSeek V3 Shows Promise
Another area where DeepSeek V3-0324 demonstrates its enhanced capabilities is in creating a weather app, similar to weather cards.
Improved, But Still Room to Grow
While the new DeepSeek V3 significantly outperforms its predecessor (R1) in this task, it still has some catching up to do compared to Claude 3.7. This highlights that while progress has been substantial, there’s ongoing development to reach the very top tier of performance. The improvement shows the potential of DeepSeek’s ongoing refinement.
DeepSeek V3-0324 Excels in Misguided Attention Eval
The “Misguided Attention” evaluation is a crucial test for assessing the reasoning capabilities of LLMs, especially when faced with misleading information.
What is Misguided Attention?
Misguided Attention is a collection of prompts designed to challenge LLMs. These prompts contain information that can potentially lead the model astray, testing its ability to reason correctly despite distractions.
V3-0324: The Best Non-Reasoning Model
The original DeepSeek V3 struggled in this evaluation. However, the V3-0324 update has dramatically improved its performance. It now ranks as the best non-reasoning model, even surpassing Sonnet-3.7 (in its non-thinking mode).
Breaking Reasoning Loops
What’s particularly impressive is that V3-0324 is solving prompts that were previously only solved by models specifically designed for reasoning (e.g., the “jugs 4 liters” problem). This suggests that the update has enabled the model to detect and break out of reasoning loops, a capability that even many reasoning models lack. The number of prompt of evaluation is at 52, thanks to community contributions.
Darker = higher number of correct responses for that specific prompt.
R1 Takes the Lead, DeepSeek Finetuning Shows Promise
In the broader evaluation, R1 (another model) has taken the lead. However, the results also highlight the significant improvement achieved by finetuning Llama-3.3 with DeepSeek traces. This demonstrates the potential of combining different models and techniques to enhance performance.
Conclusion: DeepSeek V3-0324 – A Significant Step Forward
The DeepSeek V3-0324 update represents a major advancement in the capabilities of this AI model. From matching leading models in code creativity to excelling in challenging reasoning tests, this update demonstrates a significant leap forward. While there’s always room for further improvement, DeepSeek V3-0324 is clearly a model to watch, showing impressive progress in the rapidly evolving field of AI.
