Migrating an existing app

Put the swap behind whatever flag mechanism you already have, so rolling back is a config change rather than a deploy. The rest of the client stays as it is.

from openai import OpenAI

client = OpenAI(
    base_url=(
        "https://llm11.com/v1" if settings.USE_LLM11 else "https://api.openai.com/v1"
    ),
    api_key=settings.LLM11_KEY if settings.USE_LLM11 else settings.OPENAI_KEY,
)

# Unchanged from here down.
res = client.chat.completions.create(model="auto", messages=messages)
answer = res.choices[0].message.content

modelis accepted and recorded, but triage chooses which model actually answers. If you need a specific model on a specific path, set that path’s project baseline instead of hard-coding it at the call site.

The baseline is the denominator in every savings figure the product shows you. Set it to the model your application is on right now, before the migration, or the numbers will compare against something you were never paying for.

Send somewhere between one and ten per cent through and leave it for a few days. What you are looking for is not the savings number, it is the escalation rate: that is the signal telling you whether the cheaper model is actually coping with your workload.

use_llm11 = hash(request_id) % 100 < settings.LLM11_PERCENT

# Log the receipt alongside your own outcome signal so the two
# can be joined later. The request id is the join key.
if use_llm11:
    log.info(
        "llm_call",
        llm11_request_id=res.id,
        verdict=res.model_extra["_llm11"]["verdict"],
        escalated=res.model_extra["_llm11"]["escalated"],
    )

Three numbers matter, and only one of them is the one on the marketing page.

• Escalation rate.High means triage is routing too cheap for your traffic. Raise the project’s criticality floor and it will start reaching for a better model sooner.
• Added latency. Triage is fast. Groundedness adds a scoring call. Cross-model adds a whole completion. If your p95 moves more than you can live with, the criticality floor is the dial.
• Your own quality signal. Whatever you already track, ticket reopens, edit rate, thumbs-down. Our verdict is evidence, not proof, and a migration judged only on our own metric is a migration judged by the vendor.

Flip the flag back. There is no data to export before you can leave and no format lock-in to unwind, because your prompts and responses were never stored unless you explicitly turned retention on for a project. Receipts stay available for your plan’s retention window, and the REST API will hand you all of them.

curl "https://llm11.com/api/v1/requests?projectId=$PROJECT_ID" \
  -H "Authorization: Bearer $LLM11_KEY"