UCP PlaygroundFeatures/Features
InspectorAgentDocs
Features

AI agent testing features

Protocol-level testing and observability for Universal Commerce Protocol — everything you need to debug, benchmark, and ship AI-ready stores.

26 models8 providers3 transports10 capabilities
Run agentInspect toolsView leaderboard
jump toCapabilities
01Compare

Multi-model side-by-side testing

Run up to five AI models simultaneously against the same store and query. Compare how each model navigates your catalog, handles variants, and completes checkout. See results on the leaderboard.

Compare Sessions · everlane.com · five synchronized replays
20 MODELSClaude Opus 5, Claude Sonnet 5, Claude Opus 4.8, Claude Sonnet 4.6, GPT-5.5, GPT-4o, o4-mini (Reasoning), Gemini 3.1 Pro, Gemini 3.6 Flash, Gemini 3.5 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash, Grok 4.5, DeepSeek R1 (Reasoning), DeepSeek V3.2, DeepSeek V4 Pro, DeepSeek V4 Flash, Kimi K3, Qwen 3.7 Max, and Llama 4 Maverick.
8 PROVIDERSAnthropic, OpenAI, Google, xAI, DeepSeek, Moonshot AI, Alibaba, and Meta — all routed through a single interface.
02Replay

Full session replay

Every agent session is recorded as an event stream with millisecond-precision offsets. Replay from the chat view or the raw backend tool-call timeline.

Session 01KHW0CP3V0A68… · Gemini 2.5 Pro · checkout reached in 26.8s
COMPARE UP TO 5Synchronized multi-session comparison. Watch the same query play out across different models side by side.
SHARE & EMBEDPublic share links (no login required) and embeddable iframes for docs, blogs, and slide decks.
03Transports

Three-transport coverage

Test all three UCP transport layers from a single interface. Automatic fallback between transports when one fails.

MCPJSON-RPC over HTTP. Tool calls for search, cart, checkout, and more.
RESTStandard HTTP endpoints. Automatic fallback when MCP is unavailable.
ECPEmbedded Checkout Protocol. Iframe + MessagePort handshake for secure payment flows.
04Conformance

Schema quality grading

Every Agent run grades your tool schemas from A to F — the declared contract your store hands an agent, scored before the agent even moves. Per-tool breakdown shows exactly which schema is costing you. For the declared layer across the public web, see UCP Checker. ↗

7Weighted checks
0–100Score per tool
A–FOverall grade
Checks parameter types, descriptions, required fields, array items, and nested object structure — with a specific issue list per tool.
05Funnel

Agent shopping funnel

Variant-aware funnel tracked via tool-call patterns — each session resolves to a shopping, procurement, or generic funnel, and completion is scored against that funnel’s own steps. See exactly where each model drops off and which stores block progress. Automate this with evals and collections.

25%Search
→
50%Details
→
75%Cart
→
100%Checkout
The shopping variant — one of three variant-aware funnels. Procurement (RFQ → quote → PO) and generic (discover → read → act) sessions are scored against their own steps.
Thirteen outcomes · variant-aware
purchase_completedcheckout_reachedcart_createdpo_submittedquote_receivedrfq_submittedsearch_onlyinfo_providedaction_takenread_onlyno_activityfailedabandoned
06Errors

Error trace & visibility

Every error is captured and linked to the specific tool call that produced it. Full request payload, response body, and timing for each failure.

Tracked error types
openrouter_errormodel_refusedmax_turns_exceededtoken_limit_exceededtool_call_invalidtool_call_timeoutwrong_variant_idno_checkout_urlmcp_error
07Audit trail

Store instructions visibility

Stores can inject response_instructions into MCP tool responses — behavioral prompts that tell the agent how to present products, handle pricing, or guide the conversation. UCP Playground extracts and stores these per tool call for a full audit trail.

DEBUG VIEWSee exactly what instructions each store injects, on which tool call, and how the agent responds to them.
08Cost

Token tracking

Prompt tokens, completion tokens, and total tokens recorded per session and per turn. Compare cost-efficiency across models and stores.

LEADERBOARD AGGREGATESAverage tokens and average duration per model. Identify which models burn through tokens and which are lean.
PER-TURN BREAKDOWNDrill into individual turns to see where token spikes happen — usually on large tool responses or verbose model output.
09Automation

Evals & collections

Automated shopping benchmarks across stores and models. Define multi-turn sequences, run them headlessly, and get a PDF report with funnel comparison, error analysis, and recommendations.

Example: Multi-Item Checkout Benchmark100% checkout rate across 4 sessions — Gemini 3.5 Flash vs Gemini 3.1 Pro on oakywood.shop and ugmonk.comView PDF
MULTI-TURN SEQUENCESScripted conversation flows that simulate real shopping: search, add to cart, checkout. Each turn is a separate API call with full orchestrator context.
CROSS-MODEL COMPARISONRun the same sequence against multiple models and stores. See which model reaches checkout and where each one fails.
PDF REPORTSFunnel comparison matrix, performance table, error classification, and actionable recommendations — downloadable and shareable.
CI/CD & SCHEDULINGCreate, run, and poll collections via the headless API. Set a cron schedule for weekly regression detection.
10Teams

Teams & workspaces

Create shared workspaces where your team can run sessions, benchmark stores, and review results together. Switch between personal and team context in one click.

OwnerFull control. Manage members, tokens, settings, and billing.
AdminInvite members, change roles, create and revoke API tokens.
MemberRun sessions, view team data, and use shared API tokens.
EMAIL INVITATIONSInvite teammates by email with a role. They get a signed link that expires in 7 days.
TEAM-SCOPED API TOKENSCreate API tokens scoped to your team. Every headless API call inherits the team context automatically.
USAGE DASHBOARDSessions, tokens used, checkout rate, and per-member breakdown — all aggregated for the current month.

Start testing

Point the Playground at your own store, or drive a reference target in a known posture.
Run agentInspect toolsView leaderboard
How it worksAutomated evalsDocs
Questions

Frequently asked

A free test harness for Universal Commerce Protocol. Point it at a store, call MCP or REST tools by hand in the Inspector, then hand the same target to a model in the Agent and watch it shop end to end. Every run is stored as a session you can replay or share.