Request Timing

Read per-request time to first token, upstream time, total latency and prompt-cache tokens from the usage API.

Every request AI Setu serves is recorded with how long it took and how much of your prompt came from the provider's cache. You read it back through the GraphQL API, per request.

Fields

Field	Meaning
`latencyMs`	Total time the gateway spent on the request: queueing, cache lookup, any failed attempts, and the winning upstream call. Always present.
`ttftMs`	Time to first token. Milliseconds from the start of the winning upstream attempt to its first response body byte.
`upstreamMs`	Milliseconds the winning upstream attempt took, from request start to response complete. This is the provider's own share of `latencyMs`.
`cacheReadTokens`	Prompt tokens served from the provider's prefix cache.
`cacheWriteTokens`	Prompt tokens this request wrote into the provider's prefix cache.
`failoverCount`	Failed upstream attempts before the response you received. `0` means the first attempt succeeded, or the response came from cache.

ttftMs and upstreamMs are null when there is nothing honest to report:

the response was served from the AI Setu cache (no upstream call was made),
the request failed before any response byte arrived, or
the row was recorded before these fields existed.

A null is never a 0. Treat it as "not measured" and leave it out of your percentiles.

latencyMs - upstreamMs is the time spent outside the winning attempt: your queueing, cache lookup and any failover attempts that came first.

Reading it

recentRequests returns the latest requests in a workspace, newest first:

curl https://api.aisetu.ai/graphql \
  -H "Authorization: Bearer $AI_SETU_PAT" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "query Recent($ws: ID!) { recentRequests(workspaceId: $ws, limit: 50) { requestId model provider latencyMs ttftMs upstreamMs cacheReadTokens cacheWriteTokens failoverCount cached status ts } }",
    "variables": { "ws": "'"$WORKSPACE_ID"'" }
  }'

The usage query returns the same ttftMs and upstreamMs on each event in recent, alongside the rollup for the window.

Why this is not in a response header

On a streamed response the gateway sends the HTTP headers before the first token exists, so a time-to-first-token header cannot be written: the value is not known yet when the headers flush. Covered streamed responses (OpenAI- compatible Chat Completions and Responses, Anthropic Messages) instead carry the numbers at the end of the stream: one SSE comment block before the terminal frame, : x-ai-setu-timing upstream_ms=… ttft_ms=… cached_tokens=… cache_write_tokens=… hedge=…. Its upstream_ms / ttft_ms use the same clock as upstreamMs / ttftMs here, measured from the request's upstream start (on a hedged request, the primary's dispatch). The gateway also emits the values as HTTP trailers (X-AI-Setu-Upstream-Ms, X-AI-Setu-TTFT-Ms, …), but trailers may be dropped by intermediaries; the SSE comment line is the reliable carrier. See Observability for the full format and coverage. The API above remains the source of record, and the only source for an uncovered provider, a cache hit, or a stream that was cancelled before its terminal frame.

On non-streaming responses the gateway also sends a standard Server-Timing: upstream;dur=<ms> header with the winning upstream attempt's duration, so you can read it straight from the HTTP response.

Fields

Reading it

Why this is not in a response header

On this page