Providers

AI Setu routes to upstream LLM providers in two ways: a small set of providers have dedicated adapters — gateway code that speaks that vendor's own wire protocol — and everything else that speaks the OpenAI-compatible chat-completions shape is reachable through a BYOK connection with a custom base URL, no bespoke adapter required.

Dedicated adapters

Only the OpenAI and Gemini adapters implement embeddings; sending an embeddings request to Anthropic, Bedrock, or Vertex is not supported.

Azure OpenAI is not a separate adapter — it's the openai adapter in Azure mode. Add it as a BYOK credential with your Azure resource's baseUrl, azureDeployment, and azureApiVersion; the gateway builds <baseUrl>/openai/deployments/<deployment>/chat/completions?api-version=<version> and authenticates with the resource key in an api-key header instead of Authorization: Bearer.

Bedrock takes an AWS model or inference-profile id (e.g. bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0) and calls it over AWS's Converse API with SigV4 signing — either the platform's AWS role, or a tenant-supplied AWS access-key pair when a bedrock BYOK credential is configured. Text chat only for now; tool calls are rejected with a clear error.

Vertex is one provider serving two model families off a single GCP service-account credential, distinguished by the model's publisher: Gemini-on-Vertex (vertex/gemini-2.5-pro) and Claude-on-Vertex (vertex/claude-sonnet-4-5@20250929). The gateway mints a short-lived OAuth2 token from the service account and caches it.

OpenAI-compatible providers (BYOK, custom base URL)

Any provider that speaks the OpenAI chat-completions wire shape rides the gateway's shared OpenAI-compatible client — there is no per-vendor adapter code for these. You reach them by adding a BYOK credential (a bearer API key, optionally a custom base_url) and giving it a routing label. That label becomes the <label>/<model> (or @<slug>/<model>) prefix you use in requests. See BYOK for how to add one.

This covers, among others:

• Groq, xAI, DeepSeek, Mistral — each has a default base URL compiled into the gateway, so a credential with no base_url still reaches the right vendor.
• AWS Bedrock Mantle — an OpenAI-compatible endpoint (https://bedrock-mantle.<region>.api.aws/v1) reached with a project API key over Authorization: Bearer, distinct from classic Bedrock's AWS SigV4 signing. Example: bedrock-mantle/deepseek-v3.2.
• Self-hosted or third-party OpenAI-compatible endpoints — vLLM, Ollama, Together, OpenRouter, or anything else that implements the same wire shape. These have no built-in default base URL or dedicated kill-switch; you supply the full endpoint when you add the credential.

// packages/admin-sdk — add a self-hosted vLLM endpoint as a BYOK credential.
// The slug you choose becomes the routing prefix.
await admin.providerCredentials.upsert({
  provider: 'openai',
  slug: 'my-vllm',
  apiKey: endpointKey,
  baseUrl: 'https://llm.internal.example.com/v1',
});

const res = await client.chat.completions.create({
  model: '@my-vllm/llama-3.3-70b',
  messages: [{ role: 'user', content: 'Hello!' }],
});

Credential shapes

Every BYOK base_url is validated against SSRF at credential-create time and re-checked at dial time: https only, private/loopback/link-local/CGNAT ranges blocked, DNS-rebind guarded.

Where to configure providers

The console's Settings → Providers & routing page (two tabs, "By model" and "By provider") shows your org's connection pool, lets you enable or disable a provider per workspace, and lets you drag-reorder which provider wins for a given model. See Routing and BYOK for the mechanics behind that UI.