Providers

AI Setu routes to upstream LLM providers in two ways: a small set of providers have dedicated adapters — gateway code that speaks that vendor's own wire protocol — and everything else that speaks the OpenAI-compatible chat-completions shape is reachable through a BYOK connection with a custom base URL, no bespoke adapter required.

Dedicated adapters

Only the OpenAI and Gemini adapters implement embeddings; sending an embeddings request to Anthropic, Bedrock, or Vertex is not supported.

Azure OpenAI is not a separate adapter — it's the openai adapter in Azure mode. Add it as a BYOK credential with your Azure resource's baseUrl, azureDeployment, and azureApiVersion; the gateway builds <baseUrl>/openai/deployments/<deployment>/chat/completions?api-version=<version> and authenticates with the resource key in an api-key header instead of Authorization: Bearer.

Bedrock takes an AWS model or inference-profile id (e.g. bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0) and calls it over AWS's Converse API with SigV4 signing — either the platform's AWS role, or a tenant-supplied AWS access-key pair when a bedrock BYOK credential is configured. Text chat only for now; tool calls are rejected with a clear error.

Which Bedrock id you send matters for both reachability and price:

• Raw in-Region ids are not invocable on demand for the current Claude models. Their AWS model cards list no in-Region endpoint, so bedrock/anthropic.claude-haiku-4-5-20251001-v1:0 is not something Bedrock serves on demand — use an inference profile.
• global. is what AI Setu dispatches for a bare catalog model. It is the base price, has the widest region pool, and is the only profile available from ap-south-1 / ap-south-2 (Mumbai and Hyderabad publish no geographic profile for these models).
• Geographic profiles (us., eu., au., jp.) cost 10% more. Anthropic's pricing page puts a 10% premium on regional and multi-region endpoints over global ones for Claude Sonnet 4.5, Haiku 4.5, Opus 4.5 and later; AWS documents the same gap as global inference saving roughly 10% over geographic. AI Setu's rate card reflects that, so bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0 bills 10% above bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0. Choose a geographic profile when data residency requires it: geographic routing stays within a geographic boundary (US, EU, APAC), while global may serve from any supported AWS commercial Region worldwide.

Vertex is one provider serving two model families off a single GCP service-account credential, distinguished by the model's publisher: Gemini-on-Vertex (vertex/gemini-2.5-pro) and Claude-on-Vertex (vertex/claude-sonnet-4-5@20250929). The gateway mints a short-lived OAuth2 token from the service account and caches it.

OpenAI-compatible providers (BYOK, custom base URL)

Any provider that speaks the OpenAI chat-completions wire shape rides the gateway's shared OpenAI-compatible client — there is no per-vendor adapter code for these. You reach them by adding a BYOK credential (a bearer API key, optionally a custom base_url) and giving it a routing label. That label becomes the <label>/<model> (or @<slug>/<model>) prefix you use in requests. See BYOK for how to add one.

This covers, among others:

• Groq, xAI, DeepSeek, Mistral — each has a default base URL compiled into the gateway, so a credential with no base_url still reaches the right vendor.

• AWS Bedrock Mantle — reached with a project API key over Authorization: Bearer, distinct from classic Bedrock's AWS SigV4 signing. It serves two model families on two base paths, chosen by the model-id prefix:

  • openai.* and the open-weight models → https://bedrock-mantle.<region>.api.aws/openai/v1 (OpenAI-compatible). Example: bedrock-mantle/deepseek-v3.2.
  • anthropic.* → https://bedrock-mantle.<region>.api.aws/anthropic, the Anthropic Messages API. Example: bedrock-mantle/anthropic.claude-haiku-4-5 — note the id carries no date suffix, unlike classic Bedrock.

  The split matters for cost: only the Anthropic family prompt-prefix-caches. A long static prefix is reused there even when the trailing content and the user turn change; on the OpenAI family nothing is reused unless the request is byte-identical. See Conversational Caching.

• Self-hosted or third-party OpenAI-compatible endpoints — vLLM, Ollama, Together, OpenRouter, or anything else that implements the same wire shape. These have no built-in default base URL or dedicated kill-switch; you supply the full endpoint when you add the credential.

// packages/admin-sdk — add a self-hosted vLLM endpoint as a BYOK credential.
// The slug you choose becomes the routing prefix.
await admin.providerCredentials.upsert({
  provider: 'openai',
  slug: 'my-vllm',
  apiKey: endpointKey,
  baseUrl: 'https://llm.internal.example.com/v1',
});

const res = await client.chat.completions.create({
  model: '@my-vllm/llama-3.3-70b',
  messages: [{ role: 'user', content: 'Hello!' }],
});

Credential shapes

Every BYOK base_url is validated against SSRF at credential-create time and re-checked at dial time: https only, private/loopback/link-local/CGNAT ranges blocked, DNS-rebind guarded.

Where to configure providers

The console's Settings → Providers & routing page (two tabs, "By model" and "By provider") shows your org's connection pool, lets you enable or disable a provider per workspace, and lets you drag-reorder which provider wins for a given model. See Routing and BYOK for the mechanics behind that UI.