BudgetAI Router
How BudgetAI chooses the cheapest capable model for every turn, then verifies before spending more.
The BudgetAI Router decides which model tier handles each turn. Its goal is simple: keep easy turns cheap and reserve expensive models for work that genuinely needs them.
How routing works
Routing runs on your own device before any model is called.
Read the message
The full turn is classified by difficulty, risk, context length, and intent.
Extract signals
Local rules flag debug work, strict output formats, long context, and multi-step agentic tasks.
Judge the tier
A small on-device model selects the lowest tier that can complete the work reliably.
Verify, then escalate
The cheap answer is checked against the contract (tests, schema, or a checklist). Only failures climb to a pricier model — so you pay for capability, not for guessing.
Run the model
The chosen tier maps to a concrete provider profile for that turn.
Tiers
| Tier | Meaning | Typical work |
|---|---|---|
R0 | Trivial | Greetings and one-liners |
R1 | Simple | Routine edits and focused tasks |
R2 | Hard | Refactors and non-trivial debugging |
R3 | Critical | Production and cross-service work |
Strategies
BudgetAI ships two routing strategies:
recommended— an on-device model (embeddings + gradient-boosted classifier) that runs with no network call. This is the default.llm_judge— a small LLM call tiers each turn. Use this when the local model bundle is not present, or point it at a local Ollama / LM Studio endpoint.
budgetai configure router --router recommended
# or
budgetai configure router --router llm_judgeGuardrails
Routing stays stable across a conversation:
- Anti-downgrade — recent higher-tier turns stay warm, protecting continuity and prompt-cache reuse.
- Sticky tier — short follow-ups inherit the active workstream instead of dropping context.
- Complaint up — a failed or rejected answer raises the next turn instead of repeating the same attempt.
When the local router bundle is absent, BudgetAI degrades gracefully: it logs a warning and pins every turn to the default tier instead of failing.