Routing

One interface for every model

Provider SDKs disagree about messages, tool calls, streaming, structured output, reasoning, errors, and usage. Plural translates those differences into one request and response contract. Your application chooses a catalog id; the rest of the call stays the same.

from plural import Plural, Message

client = Plural(base_url="https://api.pluralintel.com/v1")

def answer_ticket(model: str, ticket: str) -> str:
    response = client.chat(
        model=model,
        messages=[
            Message(
                role="user",
                content=f"Reply to this support ticket: {ticket}",
            )
        ],
    )
    return response.text or ""

ticket = "I was charged twice for order A123."

print(answer_ticket("openai/gpt-5.6-luna", ticket))
print(answer_ticket("anthropic/claude-sonnet-5", ticket))

The support function does not know whether OpenAI or Anthropic serves the ticket. That makes trying a new model a configuration change instead of an integration project. The normalized response always exposes text, model, provider, usage, and latency_ms.

Keep capabilities portable

The same model swap works for normal responses, streaming, tools, structured output, and reasoning. Plural accepts one OpenAI-shaped tool definition and translates it for each host. Structured JSON returns as message content even when a provider implements it with a forced tool. Streaming text always arrives on chunk.delta.content.

Add fallbacks without changing the caller

Put acceptable substitutes in models. Plural tries the primary first, then falls through the list for retryable provider failures. The caller still receives one ChatResponse, and the actual model and provider remain visible.

response = client.chat(
    model="openai/gpt-5.6-luna",
    models=[
        "anthropic/claude-sonnet-5",
        "google/gemini-3.7-flash",
    ],
    messages=[
        Message(
            role="user",
            content="I was charged twice for order A123.",
        )
    ],
)

print(response.model)     # model that answered
print(response.provider)  # provider that served it
print(response.text)

Bound retries and spend

Set max_retries for transient host failures and max_cost_usd to reject a request before it exceeds its budget. A multi-host model may route through different providers or regions; the request is billed at the endpoint actually used, with no Last Labs routing markup.

Routing policies

Most applications can stop at model ids and ordered fallbacks. Use a routing policy when you want Plural to reorder candidates. LeastCost keeps the primary first and sorts fallbacks by catalog price. LowestLatency uses provider latency metadata. A custom RoutingPolicy.select() can apply your own rule or a learned router without changing the support function above.

from plural import Plural
from plural.routing import LeastCost, LowestLatency

gateway = "https://api.pluralintel.com/v1"
cost_client = Plural(base_url=gateway, policy=LeastCost())
latency_client = Plural(base_url=gateway, policy=LowestLatency())