Skip to content
SlashClone
DiscoverUse in ClaudeSubmit a link
Sign in
Made by@agentcosts

Consumption guardrails

@agentcosts/consumption-guardrails

Set and check consumption guardrails for an AI product: per-customer daily and monthly spend caps, month-end forecasts, spend spike rules, runaway agent runs, and failure rate spikes, plus a runbook for what to do when each alert fires. Runs on usage files; exit code 2 on critical alerts for cron or CI.

SkillDevOpsv1.0.0 (latest)Tested
Use in Claude Save Like Clone Comment
Activity
New: be the first to try it
  1. 1---
  2. 2name: consumption-guardrails
  3. 3description: Design and check consumption guardrails for an AI or agentic product. Sets per-customer daily and monthly spend caps, month-end forecasts, spend spike (anomaly) rules, limits on cost and calls per agent run, and failure rate alerts; checks usage against them; and writes a runbook for each alert. Use when the user asks for "spend alerts", "usage caps", "stop runaway agents", "cost anomaly detection", "a customer blew through their budget", "per-customer limits", or wants a daily cost check in cron or CI.
  4. 4license: MIT
  5. 5---
  6. 6
  7. 7# Consumption guardrails
  8. 8
  9. 9Agent products have a failure mode normal SaaS does not: one bad loop or one enthusiastic customer can burn a month of margin in an afternoon. Guardrails are simple numbers, checked often, with a plan for each alert. This kit sets the numbers from your own usage, checks them, and writes the runbook.
  10. 10
  11. 11## Inputs
  12. 12
  13. 131. **Usage** in the shared schema (`timestamp, customer, task, run_id, model, tokens..., cost_usd, outcome`), as fresh as possible. Use `llm-usage-normalizer` for provider exports.
  14. 142. **Config** `data/guardrails.json` (start from `examples/guardrails.json`):
  15. 15
  16. 16 ```json
  17. 17 {
  18. 18 "warn_at": 0.8,
  19. 19 "defaults": { "daily_cap_usd": 6, "monthly_cap_usd": 100, "run_cost_cap_usd": 3, "max_calls_per_run": 25 },
  20. 20 "customers": { "cobalt-labs": { "monthly_cap_usd": 60, "daily_cap_usd": 4 } },
  21. 21 "anomaly": { "multiplier": 3, "baseline_days": 7, "min_usd": 1 },
  22. 22 "failure_rate": { "max": 0.4, "min_runs": 5 }
  23. 23 }
  24. 24 ```
  25. 25
  26. 26## Steps
  27. 27
  28. 281. **Set limits from data, not guesses.** Look at 30 days of usage: each customer's normal and heaviest day, the p95 cost and call count per run for each task, and what each customer pays. Propose:
  29. 29 - Monthly cap: the spend at which that customer's gross margin hits the floor the user accepts (revenue x (1 - target margin)), or their plan's included usage.
  30. 30 - Daily cap: about 3x a normal heavy day.
  31. 31 - Cost per run: 3x the p95 of the most expensive task. Calls per run: 2x the p95 calls.
  32. 32 Show the proposal as a table and write `data/guardrails.json` once the user agrees.
  33. 332. **Check:**
  34. 34
  35. 35 ```bash
  36. 36 node scripts/guardrails.mjs data/usage.csv --config data/guardrails.json --prices data/llm-prices.csv --out out/guardrails
  37. 37 ```
  38. 38
  39. 39 `--as-of 2026-10-27` checks a past day (useful to test the rules on a known incident). `--fail-on-alert` exits with code 2 on any critical alert, for a scheduled job or CI step.
  40. 403. **Report** from `out/guardrails.md`: critical alerts first, with the customer, the rule, and the numbers. For runaway runs, list run IDs so engineering can trace them.
  41. 414. **Write the runbook** `out/guardrails-runbook.md` from `templates/runbook.md`: fill in the limits table with the agreed numbers and why, who owns each alert, and the action for each. Ask the user what the product should do at a cap (notify only, throttle, queue, or pause non-urgent tasks) and who can raise a cap.
  42. 425. **Make it routine.** Suggest running the check on a schedule after usage is exported (daily is a good start; hourly for high-volume products) and sending the markdown to the team channel. Do not set up schedules or notifications yourself; give the user the command.
  43. 436. **Tune.** After a week, look at alerts that fired: if one rule fires daily with no action, its threshold is wrong. Change the config, not the code.
  44. 44
  45. 45## Rules the script applies
  46. 46
  47. 47| Rule | Fires when | Severity |
  48. 48| --- | --- | --- |
  49. 49| Daily cap | spend on the as-of day > `daily_cap_usd` | warning, critical over 2x |
  50. 50| Monthly cap | month to date > `monthly_cap_usd` | critical |
  51. 51| Monthly forecast | month to date pace > cap | warning |
  52. 52| Near cap | month to date > `warn_at` x cap | info |
  53. 53| Spend spike | day > `multiplier` x median of the prior `baseline_days` and at least `min_usd` (customers with less history than `baseline_days` are skipped) | warning, critical over 2x the multiplier |
  54. 54| Runaway runs | a run over `run_cost_cap_usd` or `max_calls_per_run` | warning, critical at 3+ runs |
  55. 55| Failure rate | failed share > `max` with at least `min_runs` runs that day | warning |
  56. 56
  57. 57## Guardrails for this kit
  58. 58
  59. 59- The script only reports. It never throttles, blocks, or changes anything in the product or the provider account.
  60. 60- Do not lower a paying customer's limits or recommend cutting them off without the account owner in the loop.
  61. 61- Costs depend on the price table. Make sure prices were checked against each provider's current pricing.
Sign in to comment on a line