Observability

Every request through the gateway is timed in three places: the whole gateway request (latency_ms), the upstream provider call (upstream_ms), and the moment the provider's first response byte arrived (ttft_ms, time to first token). Your own distributed trace also continues through the gateway: an inbound traceparent header is forwarded onto the provider request.

Timings

Only the attempt that produced your response is timed. A failed-over attempt or a losing hedge does not contribute. Both upstream fields are absent on a cache hit and whenever no body byte arrived, so a missing value means "no upstream call", not zero.

Where the numbers show up

• Buffered responses carry a standard Server-Timing header you can read in browser dev tools or any HTTP client:

  Server-Timing: upstream;dur=812, ttft;dur=241

• Streamed responses flushed their headers at the first byte, before the values existed, so no timing header can be attached to a stream. Instead the gateway writes one SSE comment block (the comment line and a blank line) at the end of the stream: right before the terminal data: [DONE] frame on Chat Completions, after the response.completed (or response.incomplete / response.failed) event on the Responses API, and right before event: message_stop on the Messages API:

  : x-ai-setu-timing upstream_ms=676 ttft_ms=440 cached_tokens=4665 cache_write_tokens=139 hedge=none
  
  data: [DONE]

  SSE parsers ignore comment lines, and the terminal frame stays a block of its own, so nothing changes for a client that does not look for it. To read it, match blocks starting with : x-ai-setu-timing and split the rest into key=value fields.

  upstream_ms and ttft_ms are measured on the same clock as the usage record: from the request's upstream start to the terminal frame and to the serving attempt's first body byte. On a hedged stream that start is the primary's dispatch, not the hedge's, so hedge=hedge reports include the hedge delay, exactly like ttftMs in the usage record. The usage record's upstreamMs ends when the stream closes rather than at the terminal frame, so it can read a millisecond or so higher. cached_tokens / cache_write_tokens are the provider's prompt-cache counts for this call and are omitted when the stream carried no usage frame; hedge is none, primary or hedge (see hedging).

  The same values are also emitted as HTTP trailers (X-AI-Setu-Upstream-Ms, X-AI-Setu-TTFT-Ms, X-AI-Setu-Cached-Tokens, X-AI-Setu-Cache-Write-Tokens, X-AI-Setu-Stream-Hedge), only on a stream that got the report. Trailers may be dropped by intermediaries (load balancers, proxies, browser fetch); the SSE comment line is the reliable carrier.

  Coverage: OpenAI-compatible Chat Completions and Responses streams (OpenAI, Bedrock Mantle GPT models, Groq, xAI, DeepSeek, Mistral) and Anthropic Messages streams (/v1/messages). Bedrock, Gemini and Vertex streams, the Anthropic chat-completions translation, and exact-cache replays carry no report yet; the usage record has their timings. A cancelled or broken stream has no terminal frame and therefore no report either. Turn the report off with GATEWAY_STREAM_TIMING=off.

• Usage and request logs store both fields on every request. They are exposed in the console's recent-requests view and on the GraphQL UsageEvent and RecentRequest types as ttftMs and upstreamMs.

Trace-context propagation

The gateway speaks W3C Trace Context. Send a traceparent (and optionally tracestate or baggage) header and the same trace id continues onto the provider request, so a span recorded on your side, a span at the gateway, and the provider's own timing line up under one trace.

curl https://gateway.aisetu.ai/v1/chat/completions \
  -H "Authorization: Bearer $AISETU_API_KEY" \
  -H "traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01" \
  -H "content-type: application/json" \
  -d '{"model":"gpt-5.6","messages":[{"role":"user","content":"hi"}]}'

Propagation is always on. Nothing is required on your side beyond sending the header, and no header means the gateway starts a fresh, non-recording context. Spans from the gateway itself are exported only when the operator configures an OpenTelemetry collector; your trace ids are forwarded either way.