Schema quality
Seven weighted checks that grade how well a store's tool schemas guide a model. Each tool, and the store overall, gets a score from 0 to 100 and a grade from A to F.
A model can only shop as well as the schemas let it. When a tool has no required fields, a model guesses which parameters are mandatory. When a parameter has no description, the model invents its meaning. Most runs where "the agent did something odd" trace back to a schema that left the model guessing. Schema quality measures that guidance before a model ever runs.
The seven checks
Each tool's inputSchema is scored against seven checks. Checks that cover several parameters give partial credit: three of four parameters described earns three quarters of that check's points.
| Check | Points | Why it matters |
|---|---|---|
Has an inputSchema | 10 | Without one, models may send arbitrary parameters. No schema stops scoring here. |
| Has properties | 15 | An empty schema tells the model nothing about what to send. |
Declares required fields | 10 | Models won't know which parameters are mandatory. |
| Every property has a description | 20 | The heaviest check: descriptions do more than anything else to steer a model. |
| Every property has an explicit type | 15 | Untyped parameters invite wrong shapes. |
Array properties define items | 15 | Without items, models can't know what goes inside the array. |
| Object properties define nested properties | 15 | An object with no declared structure is a guess the model has to make. |
Grades
| Grade | Score |
|---|---|
| A | 90–100 |
| B | 75–89 |
| C | 60–74 |
| D | 40–59 |
| F | 0–39 |
The overall score is the average of the tool scores. Every issue names the parameters involved, so the list of issues is also your list of fixes.
The API returns the grade as schema_quality: in the answer to POST /api/v1/inspect/tools, and on each run from GET /api/v1/sessions/{ulid}. One grader produces it, so the same tool list always gets the same grade. Schema quality is a measured fact about the store's published schemas, not a judgement of the store. It grades UCP tools only; the tools a store's website registers (WebMCP) are not graded.
Why letter grades here and not elsewhere
Schema quality is a weighted scorecard over many small criteria, which is where letter grades are honest (the same reason TLS scanners grade configurations A to F). Questions about behaviour with a few distinct outcomes, like identity enforcement, get words instead. A grade on a yes-or-no question would read as certification, which the Playground doesn't do.
Improving a grade
- Add
required: the most common miss, and cheap to fix. - Describe every parameter: write for a model that has never seen your store, with units, formats and allowed values.
- Type everything, including array
itemsand nested object properties: the deep structure is where models invent shapes. - Grade again after each change: fetch the tools again and the grade is worked out fresh; nothing is cached.
Something wrong on this page?