Funnel & Outcomes
The funnel measures how far a session progressed; the outcome classifies how it ended. Both are variant-aware: the harness resolves one of three funnels per session (shopping, procurement, generic) and scores completion against that funnel's own steps. Lodging runs read the shopping funnel in booking terms.
Funnel variants
A session is scored against the funnel that matches what actually happened, not a fixed retail template:
| Funnel | Steps | When it applies |
|---|---|---|
| Shopping | Search Products → View Details → Add to Cart → Checkout | The default for stores that declare a UCP shopping service. |
| Procurement | Search Catalog → Request Quote → Receive Quote → Submit PO | A shopping session switches to this funnel when procurement steps (RFQ, quote, PO) were hit — inferred from the tools called. |
| Lodging | Search Stays → Hold Room → Book → Booked | Stores that declare the draft dev.ucp.lodging service at connect. Scored exactly like Shopping, on the same stored step keys (search, cart, checkout, purchase); only the step names and outcome labels read as a booking. A held room is a booking session the store created; Booked is a confirmed reservation. |
| Generic | Discover Tools → Read / Query → Action | MCP servers that declare no recognized commerce service. Discovery-shaped, not a purchase funnel — the harness never asserts a sales workflow the server didn't claim. |
Completion rate is denominator-correct
The completion rate is the fraction of the resolved funnel's steps that were completed — a three-step generic funnel scores in thirds, a four-step shopping funnel in quarters. There is no fixed "25% per step" grid.
One deliberate exception: a terminal purchase or booking is 100% regardless of skipped middle steps. A flight booking legitimately goes search → complete with no details or cart hop; that session is complete, not half-done.
How steps are detected
Step detection matches the tool names the agent called against per-step pattern lists. Calling a matching tool once (or many times — repeats don't stack) marks that step complete:
{
"search": ["search_shop_catalog", "search_products", "search_hotels", "search_properties", "..."],
"details": ["get_product_details", "get_hotel_details", "..."],
"cart": ["add_to_cart", "create_checkout", "create_booking_session", "update_booking_session", "..."],
"checkout": ["complete_checkout", "complete_booking", "complete_booking_session", "..."],
"rfq": ["create_rfq"],
"quote": ["get_quote"],
"po": ["submit_po"]
}The patterns cover retail, travel/booking, and procurement tool vocabularies, so a hotel search marks the Search step exactly like a catalog search does.
The outcome ladder
Beyond funnel progress, every session gets one outcome label — the highest rung it reached. Commerce sessions (shopping and procurement) classify top-down:
| Outcome | Means | Reads as |
|---|---|---|
purchase_completed | A confirmed purchase or booking — the terminal success. | pass |
checkout_reached | The store's response showed a checkout: a checkout URL, or a checkout status or order it returned. On a store-page (WebMCP) run, the run reached the store's checkout page. | pass |
cart_created | Items in a cart, no checkout. | partial |
po_submitted | Procurement: a purchase order was submitted — the procurement terminal success. | pass |
quote_received | Procurement: a quote came back, no PO. | partial |
rfq_submitted | Procurement: an RFQ went out, no quote yet. | partial |
search_only | Searched the catalog, went no further. | partial |
info_provided | Answered with information (policies, FAQs) without shopping actions. | info |
failed | An actual error stopped the session — reserved for errors, never for "didn't buy". | fail |
abandoned | Stopped mid-task without an error. | unknown |
Lodging sessions use the same ladder under booking names: purchase_completed reads Booked, checkout_reached reads Booking Attempted, and cart_created reads Room Held. The stored values, and API filters on them, don't change.
Generic sessions use an activity-shaped ladder instead — an agent exploring an unknown server isn't necessarily trying to check out, so "no checkout" is not a failure:
| Outcome | Means | Reads as |
|---|---|---|
action_taken | A state-changing tool was invoked. | pass |
read_only | Discovery or read/query tools only. | partial |
no_activity | No tool activity — neutral, not a failure. | neutral |
Common funnel drops
Where shopping sessions stall usually diagnoses a UCP implementation issue:
- Search to Details — the search response lacks product handles or IDs the agent can pass onward. Fix: ensure search results include actionable identifiers.
- Details to Cart — variant selection is the most common failure point: the agent can't map user preferences (color, size) to a variant ID. Fix: return clear variant structures with human-readable option names.
- Cart to Checkout — the cart response is missing a checkout URL, or checkout requires payment data the agent can't provide. Fix: include checkout links in cart responses and support test payment methods.
checkout_reached means the store showed a checkout — not that an order exists. The model saying it checked out doesn't count. A confirmed purchase or booking classifies as purchase_completed, which in practice you'll see on demo and reference targets; most production stores hand off before payment.
Store-page (WebMCP) runs
A run through the tools a store's own page registers gets an outcome like any other session. It stops at the store's checkout and never pays, so its highest outcome is checkout_reached. See WebMCP: the store page.