DeepSeekDeepSeek V4 Pro
DeepSeek's V4-generation flagship, succeeding V3.2 — which held the highest checkout rate in our prior benchmark (68%) with an autonomous, push-through-errors style. Newly added to the lineup; playground benchmarking is in progress, so no V4 completion metrics yet.
- Avg tokens per run
- 118,450
- Avg duration
- 68.2s
- Turns to checkout
- 15.8
Shopping score
18/100Weak
Scores come from runs on UCP stores, not estimates: every figure is a tool call, a checkout attempt or a store’s answer.
Shopping score breakdown
- Checkout rate40%7.5%
- Cart rate20%35%
- Search rate10%77.5%
- Turn efficiency15%0%
- Token efficiency15%0%
22.5% of runs failed with errors.
Top stores
Development stores are hidden.
| Store | Checkout | Cart |
|---|---|---|
| store hidden | 50% | 50% |
| store hidden | 9.5% | 57.1% |
| store hidden | 0% | 50% |
| everlane.com | 0% | 16.7% |
| singon.com.np | 0% | 0% |
| allbirds.com | 0% | 0% |
| store hidden | 0% | 66.7% |
| store hidden | 0% | 50% |
| store hidden | 0% | 66.7% |
Known issues
No known issues documented for this model yet.
Token usage
- Avg per run
- 118,450
- Average of all models
- 77,769
- Against the average
- +52%
- Median (p50)
- 52,774
- p90
- 300,718
Prompt and completion
Prompt 116,427 (98%)Completion 2,023 (2%)
Daily average tokens per run, last 30 days
Oct 17Total 137,799Prompt 135,441Completion 2,358
Range
- Min
- 2,711
- p25
- 19,570
- p50
- 52,774
- p75
- 153,569
- p90
- 300,718
- Max
- 666,190
Test DeepSeek V4 Pro on your store
Start a run to see how DeepSeek V4 Pro handles your store’s checkout, call by call.
Compare with other models
Qwen 3.8 Flash 47.1%Llama 3.3 70B 39.3%Gemini 3.5 Flash 37.8%Claude Sonnet 4.6 34.6%Grok 4.3 26.3%Gemini 2.5 Flash 22.5%Claude Opus 4.8 22.3%DeepSeek V3.2 20.4%DeepSeek V4.1 Flash 20%Gemini 2.5 Pro 19.5%Gemini 3.1 Pro 19%GPT-4o 14.7%Grok 3 Mini (Reasoning) 11.1%Claude Sonnet 5 10.8%GPT-5.5 10.4%Claude Opus 5 9%o4-mini (Reasoning) 8.9%Grok 4.5 7.5%DeepSeek V4 Flash 7%Gemini 3.8 Flash 5.6%Gemini 3.6 Flash 5.2%Qwen 3.7 Max 4.8%DeepSeek R1 (Reasoning) 3.4%Kimi K3 3%Llama 4 Maverick 2.2%Gemini 3.7 Flash 0%GPT-6 Luna 0%Muse Spark 1.3 0%QwQ 32B (Reasoning) 0%All models
Rates are checkout rates. Full comparison view