Prompts

Managed prompts let you store a reusable prompt template on AI Setu instead of embedding it in every caller — versioned, labeled for promotion, and attributed in your usage/cost data. A template version freezes the whole request shape: the message array (with {{variable}} placeholders), optional tool schema, declared variables, and model config (provider/model, temperature, and other parameters) — not just text.

Calling a managed prompt

Two ways to attach a managed prompt to a completion — pick whichever fits your caller.

Dedicated path, when the caller has nothing to send but variables:

curl https://gateway.aisetu.ai/v1/prompts/welcome-message@production/completions \
  -H "Authorization: Bearer $AI_SETU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "variables": {"customer_name": "Aarav"}
  }'

Header attach, when an existing /v1/chat/completions caller wants to opt a live turn into a managed prompt without changing its request path — the template's rendered messages are prepended ahead of whatever messages you send:

curl https://gateway.aisetu.ai/v1/chat/completions \
  -H "Authorization: Bearer $AI_SETU_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-AI-Setu-Prompt-Ref: classify-ticket@production" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is my order status?"}],
    "promptVariables": {"ticket": "12345"}
  }'

// @ai-setu/client
const res = await client.prompts.completions('classify-ticket@production', {
  variables: { ticket: '12345' },
  model: 'gpt-4o-mini',
});

Both paths run through the exact same routing, failover, caching, guardrail, and billing pipeline as a plain /v1/chat/completions call — a managed prompt doesn't opt out of anything documented elsewhere in these docs.

Referencing a prompt — slug, @label, @version

Labels (production, staging, …) are movable pointers you re-point to promote or roll back a prompt without touching any caller. Note that a label re-point can take up to about 30 seconds to reach live traffic on the gateway, since resolved refs are cached briefly for latency.

Variables and rendering

Variable values are substituted as plain strings — never evaluated as code, SQL, or a template directive. Substitution is single-pass: a variable value that itself contains {{...}} (including block syntax like {{#if ...}}) is not re-expanded, so a variable's content can't inject another variable, a partial, or a block. Rendered prompts are never logged or persisted; only the template id, version, and (if applicable) experiment id are recorded against your usage — never the rendered text.

If your template composes a static prefix (identity, tone, safety instructions) followed by dynamic content (customer name, retrieved context, the live turn), keep the dynamic pieces at the end of the message array. Vendors like Anthropic and OpenAI cache on the longest byte-identical prefix of a request — placing a variable early breaks that prefix and collapses your prompt-cache hit rate.

Prompt Studio and the Playground check this for you. While you edit (and on the version you saved last), a prompt-cache prefix banner lists every {{variable}}, {{>partial}} or {{#if}}/{{#each}} block that sits before static content, flags variables inside the system prompt as errors, and shows an estimate of how many tokens of your prompt stay cacheable. The usual fix is to move variables into the last user message or a trailing system message. The same check is available as the lintPromptVersion GraphQL query, and savePromptVersion returns it as prefixLint.

Conditional and repeated sections

Templates can switch sections on and off and repeat a section for each item of a list, so the whole prompt (not just its skeleton) lives in AI Setu:

Truthy means true, a non-empty string, a non-zero number, a non-empty list, or an object with at least one key. Blocks nest up to 8 levels. A block tag on its own line removes that whole line, so turning a section off leaves no blank lines behind.

Variables have a type: string, number, boolean, list (a JSON array) or object (a JSON object). Send list and object values as JSON in variables:

You are Saar's assistant.
{{#if journey_onboarding}}
Walk the user through these steps:
{{#each onboarding_steps}}
{{@index}}. {{this.title}}
{{/each}}
{{/if}}
{{#unless capabilities.payments}}
Do not offer to take payments.
{{/unless}}

{
  "variables": {
    "journey_onboarding": true,
    "onboarding_steps": [{ "title": "Verify your number" }, { "title": "Pick a plan" }],
    "capabilities": { "payments": false }
  }
}

Rules that keep rendering safe and predictable:

• Saving a version checks the template: an unclosed or mismatched block, an unknown block, or a variable used anywhere (including inside a block) that isn't declared returns a 400 with the line and column. {{#each}} needs a variable declared as list.
• A variable with no value and no default is an error, even in an {{#if}}. Give optional flags a default such as false.
• A render may produce at most 256 KB of text, run at most 10,000 loop iterations in total, and take at most 50 ms; past any limit the render fails with a 400 explaining which limit was hit.

Languages (locale)

One template can carry the same prompt in several languages. When you save a version, set its locale (a BCP-47 tag such as en, hi, ar or en-US); give the template a default locale; and set labels per locale (prod for hi and prod for en are separate pointers). Then ask for a language:

• in the ref: saar-system#hi, saar-system@prod#ar, saar-system@3#en;
• or, on the render endpoint, with a locale field in the body.

AI Setu picks, in order: the requested locale (its prod label, else its newest version), then the template's default locale, then the version without a locale. If the language you asked for has no version, the request still succeeds, and the render response says so:

{ "templateId": "…", "version": 4, "messages": [ … ], "locale": "en", "localeFallback": true }

slug@3#hi always serves version 3, and reports localeFallback: true if version 3 isn't Hindi. A label for a locale can only point at a version in that locale. Partials included by a version are picked in that version's locale too. An A/B experiment only applies to a locale request when the version it picks is in that locale.

On the gateway, send the locale as a header on /v1/prompts/{ref}/completions or alongside X-AI-Setu-Prompt-Ref on /v1/chat/completions:

curl https://gateway.aisetu.ai/v1/prompts/classify-ticket/completions \
  -H "Authorization: Bearer $AI_SETU_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-AI-Setu-Prompt-Locale: hi" \
  -d '{"variables":{"ticket":"12345"}}'

If you can't set headers, send promptLocale in the body instead (it's removed before the request reaches the model provider); the header wins if both are set. The locale must look like a BCP-47 tag: letters, digits and hyphens, at most 35 characters, otherwise the request fails with 400 before any prompt is rendered. If you also put a #locale in the ref, the two must agree.

When the language you asked for has no version and another one is served, the response carries X-AI-Setu-Prompt-Locale-Fallback: true, so a missing translation never goes unnoticed. The #locale ref qualifier (X-AI-Setu-Prompt-Ref: classify-ticket@prod#hi, or /v1/prompts/classify-ticket%23hi/completions with the # percent-encoded) keeps working, and reports a fallback the same way.

Versioning, labels, and A/B experiments

Manage templates, versions, labels, and A/B experiments from the console's Playground (/playground) — edit messages and variables, run single or side-by-side comparisons across models with per-column latency and cost, save a version, set a label, or start a weighted A/B split across versions. An active experiment on a template overrides bare-slug resolution with a weighted pick; an explicit @version or @label always bypasses the experiment.

Sticky assignment per end user

By default each request rolls the weighted pick independently, so one end user can see different variants on consecutive requests. To keep a user on one variant, send a stable key (a user id or session id) with the request, either as a header or a body field:

curl https://gateway.aisetu.ai/v1/prompts/classify-ticket/completions \
  -H "Authorization: Bearer $AI_SETU_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-AI-Setu-Prompt-Sticky-Key: user-1234" \
  -d '{"variables":{"ticket":"12345"}}'

On /v1/chat/completions with X-AI-Setu-Prompt-Ref, and on /v1/prompts/{ref}/completions, the key can also be sent in the body as promptStickyKey; the header wins if both are set. The key is a string of at most 256 characters. The same key always gets the same variant for as long as the experiment's variants and weights are unchanged; changing the split or starting a new experiment reshuffles users. Requests without a key keep the random behaviour, and an explicit @version or @label ignores the key. Because resolved refs are cached on the gateway for about 30 seconds, label changes can lag, but the sticky assignment itself is computed per request and is not affected by that cache.

Per-tenant overrides (planned)

If you run one AI Setu workspace per end customer, you will be able to keep a prompt in one platform workspace and let each customer workspace replace only the parts that differ, such as a persona name, brand facts or an escalation contact, without copying the whole template into every workspace.

The intended model:

• Your organization names one base workspace. Every other workspace in the organization can call the base workspace's templates by slug.
• In the base workspace you mark specific partials as overridable. A customer workspace saves its own text for those partials only; partials that are not overridable, such as safety rules, always come from the base.
• A request is rendered with the overrides of the workspace its API key belongs to. No extra header is needed, and a key never sees another workspace's overrides.
• Versions, labels and A/B experiments stay on the base template, so promoting a version or running an experiment covers every customer at once, and each customer's override applies to every experiment arm.

Until this ships, use one template per workspace, or pass the per-customer text as a variable.

Rendering from your runtime

To render a template's variables into concrete messages without making an inference call — useful if you want to run the request through your own client, or to fetch the pinned prompt at boot and refresh it on a timer — use the render-only path. It accepts the same gateway API key you already use for inference, so your runtime does not need a second credential:

curl https://api.aisetu.ai/v1/prompts/classify-ticket@production/render \
  -H "Authorization: Bearer $AI_SETU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"variables":{"ticket":"12345"},"stickyKey":"user-1234"}'

The response carries the rendered messages and the version that was served, plus experimentId when a running experiment picked it:

{ "templateId": "…", "version": 3, "messages": [ … ], "experimentId": "…" }

• Sticky A/B. stickyKey (a string of at most 256 characters) keeps one end user on one variant, exactly as on the gateway paths above. Leave it out for a random pick; an explicit @version or @label ignores it.
• Scope. A gateway key renders only prompts in its own workspace. You can omit workspaceId (slug refs resolve in the key's workspace); sending a different one is a 403. Revoked keys, and keys of a deleted workspace, get a 401.
• Read-only. A gateway key can render prompts and nothing else on this API: it cannot read or change templates, labels or experiments. For that, use the console or a personal access token.

The client SDK exposes the same call:

const { messages } = await client.prompts.render('classify-ticket@production', {
  ticket: '12345',
});

To compare versions or an experiment's variants on quality, score them against a dataset with an LLM judge — see Evals.