Audit Trail for Agent API Selection

An audit trail for agent API selection answers a narrow operational question: why did this agent pay this provider for this request? The answer must connect the charge to the requirements, candidates, measurements, rule, and exclusions that existed when the provider was chosen.

A normal API log starts too late. It records the winning call after the other offers have disappeared. A general agent trace may show tool invocations, but still omit the fields needed to reproduce the selection boundary. The audit trail has to join selection and execution without pretending they were one event.

Start from the paid call and walk backward

Most investigations begin with an outcome, not a decision. An operator sees a provider charge, a failed response, a retry, or an unexpected endpoint. The first task is to identify the exact execution, then follow stable references back to the selection that authorized it.

The chain should answer these questions in order:

• Which provider request produced the charge or outcome?
• Which decision identifier was attached to that request?
• What candidate set and requirement set belonged to that decision?
• Which rule selected the winner after hard constraints were applied?
• Which candidates were rejected, and what exact constraint removed each one?
• Which eligible providers remained as ordered runners-up?
• Did an override, retry, or fresh decision occur before execution?

This backward walk differs from simply recording agent provider decisions. Recording defines the object to retain at selection time. An audit tests whether that object can be connected to one real transaction later, across the systems that selected, paid, called, and retried the provider.

Use identifiers that survive every boundary

Timestamps help, but they are weak join keys. Concurrent agents can make similar calls within the same second, and retries can create several provider requests for one buying decision. A reliable trail uses explicit identifiers rather than asking an investigator to correlate events by time and guesswork.

Keep a stable decision identifier on the selection record. Carry it into the downstream provider request where the integration permits it. Add a workflow or trace identifier for the wider agent run, plus separate request and payment identifiers for events that can repeat.

These identifiers describe different scopes. One workflow may contain several decisions. One decision may lead to a failed request and a retry. A fresh decision after measurements or requirements change should receive a new decision identifier, even when it belongs to the same workflow.

Do not collapse those events into a single mutable status row. Preserve an ordered history: decision created, provider request attempted, payment accepted or rejected, response received, override recorded, or new decision requested. The sequence is more useful than a final label because it shows which facts were available before each action.

Preserve the evidence as it existed then

An audit cannot depend on today's catalogue. Provider prices, payment terms, schemas, and operating measurements can change. Re-running a comparison now does not recreate an old selection; it creates a new selection from current evidence.

The historical record should retain the original candidate identifiers, requirements, and returned decision data. That includes the chosen provider, selection rule, rejected candidates and exact failed constraints, and the ordered runners-up. It should also preserve the weights and per-dimension contributions behind any relative score.

Apiosk's 0–100 scores are relative to the candidates in one comparison. A stored total without its candidate set and arithmetic is not durable evidence. Transparent API scoring weights matter here because a reviewer must be able to see which measured dimension moved the historical order.

Missing measurements must remain missing. Apiosk drops unavailable dimensions from the weighting and names them; it does not score them zero. Result quality and provider terms are not measured anywhere. An audit trail should state those limits instead of expanding a selection record into claims the comparison never made.

Separate exclusions, alternatives, and external leads

Three kinds of non-winners can appear around discovery, but they do not have the same status.

A rejected candidate failed a hard requirement such as `max_price`, `max_latency_ms`, `min_reliability`, settlement preference, or `require_all_inputs`. Its exclusion is part of the decision evidence. An eligible runner-up passed the hard requirements and ranked below the winner under `optimize_for=price|latency|reliability|balanced`. It can support an operator override without relaxing the original contract.

An external x402 discovery is neither. Apiosk returns external hits in a separate `external_candidates` block. They are unreviewed, unmeasured, have no `candidate_id`, and cannot chain into comparison, decision, or settlement. An audit must not rewrite an external lead as if it were a rejected or eligible provider.

Keeping these categories distinct makes explainable API routing decisions useful during an incident. The reviewer can see whether an alternative was disqualified, ranked second, or never entered the reviewed selection set.

Audit the rule before judging the outcome

A poor outcome does not by itself prove that the recorded selection rule was applied incorrectly. The provider may have failed after a valid choice, fresh conditions may have appeared, or the buyer may care about a factor outside the measured dimensions.

Review the decision in two passes. First, check procedural consistency: were the stated requirements applied, were ineligible candidates removed, and did the returned weights and contributions produce the recorded order? Second, evaluate whether the buyer supplied the right requirements and optimization rule for the workload.

This separation prevents hindsight from changing the evidence. If an operator selected a runner-up, keep the automated winner and add the override as a later event with its reason. The workflow for overruling an automated provider choice preserves both the rule's output and the human action.

If facts changed before the call, record a fresh decision. Do not edit the old requirements or candidate order. An append-only sequence makes it possible to distinguish a policy problem, a data change, a provider execution failure, and a human exception.

Keep the trail useful without copying everything

An audit trail does not need hidden model reasoning or every prompt token. It needs structured evidence at the decision boundary and references to downstream events. Copying complete prompts and responses into an operational ledger can add sensitive data without improving the explanation of provider selection.

Define access and retention around the records you actually need. Keep candidate and decision fields queryable by time, workflow, provider, rule, and outcome. Redact sensitive task inputs before long-term storage. Restrict full request or response bodies separately from the compact selection record.

The test is practical: can an authorized reviewer start with one charge and reach the contemporaneous decision without relying on a model to narrate what probably happened? If the answer requires current catalogue data, timestamp matching, or rewritten summaries, the trail is incomplete.

Where Apiosk fits

Apiosk provides the selection evidence that ordinary provider logs cannot create. `GET /v1/decide?candidates={id},{id}` returns one provider, the rule that picked it, every rejected candidate with the exact constraint that removed it, and runners-up in order. Shared requirements can travel from discovery through comparison and decision.

`POST /v1/decisions` is free and unauthenticated. It records which provider was chosen and which alternatives were rejected. Your system can connect that decision record to its own workflow, request, payment, and outcome identifiers while keeping those execution events separate.

Apiosk does not make the resulting trail tamper-proof, set your retention policy, or measure result quality and provider terms. It supplies a structured decision boundary with explicit limits. If you operate an endpoint, you can list your API on Apiosk so provider selection produces evidence that a buyer can inspect when one paid call is questioned later.