Unmeasured Dimensions in API Comparison Stay Unknown

Unmeasured dimensions in API comparison are not weak scores. They are unknowns. If you run an endpoint, that distinction prevents a comparison layer from turning absent evidence about your result quality or provider terms into a negative claim.

It also protects the buyer. A high operational score should not imply that an endpoint produces the best result, fits a particular data policy or offers acceptable contract terms. Those questions need their own evidence and evaluation method.

Missing, zero and not applicable are different states

A useful comparison keeps three states separate. Zero means a dimension was measured and the candidate received the bottom value under the stated method. Missing means no usable measurement exists. Not applicable means the dimension does not belong to that candidate or request.

Collapsing those states damages the ranking in both directions. Scoring missing evidence as zero punishes a provider for the observer's data gap. Treating it as average gives the provider credit without support. Removing it without naming it hides how narrow the conclusion is.

The response should therefore carry the state beside the value. For every candidate and dimension, a buyer needs to know:

• Whether the dimension was included, omitted or not applicable.
• Which observed value entered the calculation.
• Which method produced that observation.
• Whether a sample threshold was met.
• Which weight was active after omissions.
• What additional check is required for an unknown that matters.

This is the boundary behind transparent API scoring weights. The arithmetic can be fully inspectable while the conclusion remains deliberately limited to available evidence.

Operational evidence does not establish result quality

An endpoint can respond quickly, succeed at the protocol level and still return an unusable answer. A `200` response establishes transport success. It does not establish factual accuracy, completeness, relevance or fitness for the buyer's task.

Result quality is also workload-specific. A geocoding endpoint might need address-level accuracy for delivery routing but only city-level accuracy for regional analytics. A document extractor might pass when required fields match a verified record and fail when it returns plausible but incorrect values. One generic quality number cannot carry both acceptance rules.

Measuring quality requires an evaluation design outside an ordinary provider comparison. The buyer needs representative inputs, expected outcomes or a defined rubric, and a rule for judging each output. Exact fields can use deterministic checks. Open-ended outputs may need calibrated human review or another documented evaluator. Either way, the result must identify the task set and method that produced it.

Until that work exists, result quality stays unknown. Price, input compatibility, measured latency and API success rate measurement are useful evidence, but none is a proxy for whether the returned content is correct.

Provider terms belong in a separate review

Provider terms answer a different class of question. They can govern data use, retention, service commitments, acceptable use, liability and termination. Those terms may be decisive even when the endpoint leads on every measured operating dimension.

A comparison layer cannot derive them from a price, settlement rail or successful call. It needs current source documents, a review date and an interpretation tied to the buyer's requirements. Even then, the evidence is not the same as a legal conclusion.

Keep the review concrete. Before routing production traffic, an operator may need to record:

• The terms and privacy documents that apply to the specific offering.
• The version or review date of each document.
• Required data handling and retention conditions.
• Service commitments and stated exclusions.
• Geographic, workload or acceptable-use restrictions.
• The person or policy that approved the evidence.

These fields should travel beside the provider decision, not disappear into an operational score. If a required term has not been checked, the decision should say so and follow the buyer's policy for missing evidence.

Omit the dimension and show the consequence

Apiosk drops an unmeasured dimension from the weighting and names it. It does not score the dimension zero. The remaining 0–100 score is relative to the candidates in that one comparison and reflects only the dimensions that entered the calculation.

Dropping a dimension changes the evidence base. The response should make that consequence visible through the active weights and each candidate's contribution. A score built from price and input compatibility is narrower than one that also includes enough observations for latency and success rate. Neither score can reach beyond those operational dimensions into result quality or provider terms.

The same discipline applies when performance evidence is still accumulating. Measured latency differs from advertised latency, and a listing without enough proxied calls should not receive an invented latency value. The buyer can accept the narrower evidence, require a measurement, run a test or exclude the candidate. The comparison should expose those choices rather than silently make one.

Turn important unknowns into gates

Not every unknown blocks a request. A low-risk enrichment task may proceed when provider terms have already been approved at account level. A regulated or customer-facing workload may require a fresh quality test and explicit data-handling evidence before any candidate is eligible.

The workflow should translate those needs into gates before ranking. For example, a buyer can require all declared inputs, bind a price ceiling or set a reliability floor through operational parameters. Result quality and provider terms need separate gates because Apiosk does not measure them.

A practical sequence is:

• Discover reviewed candidates that perform the required job.
• Apply hard operational requirements to the candidate set.
• Compare the dimensions for which evidence exists.
• Read every named omission before accepting the order.
• Run the buyer's task-specific quality evaluation where required.
• Check current provider terms against the buyer's policy.
• Record which evidence supported the final choice.

This makes comparing paid APIs for AI agents a staged decision instead of a claim that one score answers every procurement question. The comparison narrows the field. Buyer-specific gates establish whether the remaining provider is actually acceptable.

Where Apiosk fits

`GET /v1/compare?candidates={id},{id}` puts price, measured latency, measured success rate and input compatibility side by side. Each 0–100 score is relative to that candidate set and travels with the active weights and per-dimension contributions. Dimensions without measurements are dropped and named.

Apiosk does not measure result quality or provider terms anywhere. Its comparison response therefore cannot prove either one. `GET /v1/decide?candidates={id},{id}` returns the selected provider, the rule that picked it, rejected candidates with their exact constraints and ordered runners-up, but the decision remains bounded by the evidence supplied to the flow.

For a provider, that boundary is useful. Your endpoint is ranked on stated and observed operational facts, while unsupported conclusions stay out of the score. To enter the reviewed candidate set, list your API with its capability, input requirements and current payment terms. Buyers can then compare what Apiosk measures and apply their own quality and terms checks to the named gaps.