Funnel & Outcomes
The funnel measures how far a run got; the outcome says how it ended. Both depend on the kind of store: each run is scored against one of three funnels (shopping, procurement, generic), using that funnel's own steps. Runs on Stays stores read the shopping funnel in booking words.
Funnel variants
A run is scored against the funnel that matches what actually happened, not a fixed retail template:
| Funnel | Steps | When it applies |
|---|---|---|
| Shopping | Search Products → View Details → Add to Cart → Checkout | The default for stores that declare a UCP shopping service. |
| Procurement | Search Catalog → Request Quote → Receive Quote → Submit PO | A shopping run switches to this funnel when it reached procurement steps (RFQ, quote, PO), judged from the tools called. |
| Lodging | Search Stays → Booking session → Book → Booked | Stores that declare the draft dev.ucp.lodging service at connect. Scored exactly like Shopping, on the same stored step keys (search, cart, checkout, purchase); only the step names and outcome labels read as a booking. A held room is a booking session the store created; Booked is a confirmed reservation. |
| Generic | Discover Tools → Read / Query → Action | MCP servers that declare no recognized commerce service. A discovery funnel, not a purchase funnel: we never assume a sales workflow the server didn't claim. |
Completion rate is denominator-correct
The completion rate is the share of the chosen funnel's steps that were completed: a three-step generic funnel scores in thirds, a four-step shopping funnel in quarters. There is no fixed "25% per step" grid.
One exception, on purpose: a completed purchase or booking is 100% even when middle steps were skipped. A flight booking can rightly go from search to complete with no details or cart step; that run is complete, not half-done.
How steps are detected
Steps are detected by matching the tool names the agent called against a list of patterns for each step. Calling a matching tool once marks that step complete; calling it again adds nothing:
{
"search": ["search_shop_catalog", "search_products", "search_hotels", "search_properties", "..."],
"details": ["get_product_details", "get_hotel_details", "..."],
"cart": ["add_to_cart", "create_checkout", "create_booking_session", "update_booking_session", "..."],
"checkout": ["complete_checkout", "complete_booking", "complete_booking_session", "..."],
"rfq": ["create_rfq"],
"quote": ["get_quote"],
"po": ["submit_po"]
}The patterns cover retail, travel and booking, and procurement tool names, so a hotel search marks the Search step just as a catalog search does.
The outcome ladder
Besides funnel progress, every run gets one outcome: the highest rung it reached. Commerce runs (shopping and procurement) are classified from the top down:
| Outcome | Means | Reads as |
|---|---|---|
purchase_completed | Order placed: the checkout is completed and carries an order. | pass |
checkout_reached | The agent got the checkout ready to place the order (the call that places it was sent or held back), or the store handed it to the buyer to finish on its page (requires_escalation with a continue_url). Not an order. On a store-page (WebMCP) run, the run reached the store's checkout page. | pass |
cart_created | Items in a cart, no checkout. | partial |
po_submitted | Procurement: a purchase order was submitted, the final success for procurement. | pass |
quote_received | Procurement: a quote came back, no PO. | partial |
rfq_submitted | Procurement: an RFQ went out, no quote yet. | partial |
search_only | Searched the catalog, went no further. | partial |
info_provided | Answered with information (policies, FAQs) without shopping actions. | info |
failed | An actual error stopped the run. Kept for errors, never for "didn't buy". | fail |
abandoned | Stopped mid-task without an error. | unknown |
Runs on Stays stores use the same ladder under booking names: purchase_completed reads Booked, checkout_reached reads Booking Attempted, and cart_created reads Booking session open (a session does not mean the store holds the room). The stored values, and API filters on them, don't change.
Generic runs use a ladder of activity instead. An agent exploring an unknown server isn't necessarily trying to check out, so "no checkout" is not a failure:
| Outcome | Means | Reads as |
|---|---|---|
action_taken | A state-changing tool was invoked. | pass |
read_only | Discovery or read/query tools only. | partial |
no_activity | No tool activity. Neutral, not a failure. | neutral |
On a run's page
A run's page says how it ended, in the words of its Timeline: Searched only, Built a cart, Reached checkout, Tried to place the order or Read an order, from the furthest stage its calls reached. It says Order placed only when the store's checkout reached completed and returned the order, never from which tools were called. A run whose calls reached none of those stages takes its words from the outcome instead, for example Answered (info_provided) or Failed (failed). When a call failed, the page also says whose it was: the store's, the agent's, or ours. A store's deliberate answer, such as payment_declined or identity_required, is the store answering, and counts against no one.
Common funnel drops
Where shopping runs stall usually points to a problem in the store's UCP set-up:
- Search to Details: the search response lacks product handles or IDs the agent can pass on. Fix: make sure search results include identifiers the agent can use.
- Details to Cart: choosing a variant is where runs fail most. The agent can't map the buyer's wishes (colour, size) to a variant ID. Fix: return clear variants with readable option names.
- Cart to Checkout: the cart response has no checkout URL, or checkout needs payment data the agent can't give. Fix: include checkout links in cart responses and support test payment methods.
checkout_reached means the store returned a checkout, not that an order exists. The model saying it checked out doesn't count. Many stores hand the last step to their own page. Those runs end at checkout_reached with a buyer handoff, which is the furthest an agent can go there.
Website (WebMCP) runs
A run through the store's website (the tools its pages register with WebMCP) gets an outcome like any other run. Outside our test stores a signed-out run stops before the order, so there its highest outcome is checkout_reached. See WebMCP: the store page.
Something wrong on this page?