STACK
MENU
DOCS / THE ACCOUNT / MODELS

Models

How an agent calls any model through the account, puts a ceiling on what a call can cost, gets large answers as files, and how you read what was used. What the service is and what it costs is on Services.

Before you start

  • An agent with an account calls models on STACK's key, paid from your credit at the provider's price. You connect nothing.
  • The account needs credit: see Prepaid credit.
  • Every call carries the agent's Passport.

Call a model

prompt
Ask a small model to summarise this page in three sentences.

From an AI client the tool is stack_call_openrouter. It reaches any model and endpoint OpenRouter has. From code or the terminal:

typescript
const answer = await stack.llm.openrouter.chat.completions.create({
  model: 'openai/gpt-4o-mini',
  messages: [{ role: 'user', content: 'Summarise this page in three sentences.' }],
}, { passport, maxCostCents: 5 });
bash
npx @getstackrun/cli llm openrouter chat-completions \
  '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"Hi"}]}'
  • For any other endpoint, such as embeddings or images: stack.llm.openrouter.request("POST", "/embeddings", { passport, body }).
  • The request and the answer pass through unchanged. STACK adds no prompt and removes nothing.
  • Streaming works from code and the terminal. The tool does not stream.
  • A call can run for 30 minutes at most.

Put a ceiling on the cost

Before a call, STACK holds the most it could cost from your credit. Afterwards it charges what the provider reports and gives the rest back. A ceiling lowers what is held:

  • From code: maxCostCents. From the tool: max_cost_cents. Over the API: the header X-Max-Cost-Cents.
  • It is a whole number of cents, 1 or more.
  • A call that names no output limit is held as if it could write about 8,000 tokens. Naming max_tokens holds less.
  • The terminal commands have no flag for the ceiling.

Large answers and files

Through the tool, an answer that is a file, or text over 256 KB, is saved to the agent's drive, and the tool returns the file's id in place of the content. Over the API, send the header X-Stack-Save-Output: drive for the same. From code, stack.llm.openrouter.requestBytes() returns a file directly.

What a model returns is content from outside, and so is a file it makes.

Use your own provider key

An agent without an account, or a call to Anthropic or OpenAI directly, runs on a key you connect in the Console. The provider bills you, and STACK counts the call as one action.

typescript
await stack.llm.anthropic.messages.create(
  { model, max_tokens: 256, messages }, { passport });

await stack.llm.openai.chat.completions.create({ model, messages }, { passport });

There is no tool for these two routes. On your own key the cost ceiling is ignored, because STACK holds nothing.

Read what was used

bash
npx @getstackrun/cli llm usage summary

stack_llm_usage_list and stack_llm_usage_summary read calls by agent, model and period. From code: stack.llm.usage.list() and stack.llm.usage.summary(). Reading usage is free.

What counts as an action

Each call is one action. On STACK's key the model's cost is charged from credit at the provider's price, with no fee from STACK.

When a call is refused

  • 403 FORBIDDEN - the call carries no Passport.
  • 403 OPENROUTER_ENDPOINT_REFUSED - the endpoint manages the provider account itself (keys, credits) and is not reachable on STACK's key.
  • 402 INSUFFICIENT_BALANCE - the credit cannot cover the hold. Top up, or lower the ceiling.
  • 402 AGENT_BUDGET_EXCEEDED - the agent's budget is used up.
  • 503 PROVIDER_CAPACITY_UNAVAILABLE - STACK's key was refused by the provider. Nothing was charged.
  • An error from the provider itself is passed on with its own status.
  • 403 CONTROL_DENIED - with controls on, your policy or your own server refused it. Controls are asked as model.call, with the model and the amount held. A policy can list the models allowed and cap the cost of one call.

Reference

Every route is on LLM Gateway. A first call from the terminal is in Make a model call.

Models | Docs