Bring Your Own LLM: Running Imperal on Your Model, Your Infrastructure

What Bring Your Own LLM Actually Means

Every Imperal tenant configures its own LLM endpoint. That's the whole claim, and it's worth stating plainly before getting into the mechanics, because "bring your own LLM" gets thrown around loosely enough that it's stopped meaning much.

On Imperal it means three concrete things, and you pick which one applies to you:

• Platform default: Claude Sonnet as the primary reasoner, configured and managed by us. This is what you get if you do nothing, and it's a reasonable place to start.
• Your own API key: point Imperal at your own Claude or OpenAI account. Your operational data is isolated to your account under your key, you're not sharing an inference pool with anyone else's tenant.
• Your own endpoint: run an OpenAI-compatible server on your own infrastructure: Ollama, vLLM, llama.cpp, whatever you've already got standing up, and point Imperal at it directly.

That third option is the one that changes the conversation. It means the reasoning happens on hardware you control, in a network you control, with no round-trip to a third-party model provider at all.

Concretely, pointing a role at your own OpenAI-compatible endpoint (Ollama, vLLM, whatever you've already got running) is one config block:

# imperal.byollm.yaml
roles:
  execution:
    provider: openai-compatible
    base_url: https://llm.internal.example.com/v1
    model: llama-3.1-70b-instruct
    api_key_ref: secrets/internal-llm-key
  routing:
    provider: anthropic
    model: claude-sonnet

And it's not all-or-nothing. Imperal lets you assign different models to different roles inside the system, routing, execution, navigation, each pointed at a different endpoint if that's what your setup calls for. You might run routing on a fast local model, execution on Claude, and navigation on whatever's cheapest per token. The configuration is per-role, not a single global switch.

Why This Matters: Sovereignty, Cost, Compliance

Three kinds of teams ask about BYOLLM, and they're asking for different reasons.

The first is about data sovereignty. If your operational context, the prompts, the intermediate reasoning, the data the agent touches while doing its job, never leaves your own LLM infrastructure, you've closed off an entire category of exposure. No third-party API is in the loop. No vendor logs your queries. This matters most to teams who've already decided that certain data doesn't leave their perimeter, full stop, and were previously told that meant no agentic tooling at all.

The second is cost control. High-volume, repetitive reasoning jobs get expensive fast on hosted frontier models, especially at the kind of call volume an agent running continuously generates. Running your highest-volume or lowest-stakes calls on a local model, while keeping the expensive frontier model for the reasoning that actually needs it, is a real lever, not a hypothetical one. You decide where the cost sits.

The third is compliance. Regulated industries have a specific sentence they need to be able to say to an auditor: data never leaves our VPC. That sentence is either true or it isn't, and no amount of vendor reassurance substitutes for the architecture actually supporting it. BYOLLM, with a self-hosted endpoint, makes that sentence true and verifiable rather than a promise in a sales deck. A compliance team doesn't need to trust that a third party is handling data properly. They need to not have to ask the question.

The Model Proposes, the Kernel Decides

Here's the part that's easy to miss when BYOLLM gets pitched as a flexibility feature: swapping the model swaps the reasoner. It does not swap the part of the system that decides what actually happens.

Imperal's architecture keeps those two things separate on purpose. The model, whichever one you've configured, hosted or local, Claude or something running on your own GPU, proposes actions: it suggests, plans, drafts, routes. The kernel is what evaluates a proposed action against policy and decides whether it runs.

This is why BYOLLM doesn't require a leap of faith the way it might on a model-centric tool, where a wrong guess reaches the delete because there's nothing standing between the model's output and the action it describes. Point Imperal at a smaller, cheaper, self-hosted model and you get a less capable reasoner, possibly a worse planner, possibly one that makes more mistakes. What you don't get is a weaker enforcement layer, because the enforcement layer isn't the model's job in the first place. A hallucination can't act.

So the decision to bring your own model is really a decision about where the reasoning happens and who pays for it and who's accountable for its infrastructure, not a decision about how much you're trusting the system to behave. That part of the equation doesn't move.

Where BYOLLM Sits Today

BYOLLM, self-hosting, and on-prem deployment are grouped together as Enterprise-tier capabilities. That's a deliberate line, not an artificial upsell. Self-serve is built for teams who want to get running fast on the platform default or their own API key. Running your own inference endpoint against your own infrastructure, wiring up per-role model assignment, and getting the deployment topology right for a regulated environment is a different kind of engagement, one that involves actual setup work on both sides.

Worth being direct about this: it's a functioning capability today, not a roadmap slide. Tenants are running their own endpoints against Imperal right now. If you're evaluating this for a compliance requirement or a cost model that depends on local inference, the honest next step is a conversation, not a wishlist.

If you want to see how this fits into the runtime more broadly, how the kernel enforces policy regardless of which model is proposing actions, read how the runtime's layers work. If you're ready to talk through what BYOLLM looks like for your infrastructure and your compliance requirements specifically, talk to us about Enterprise.