Made by@agentcostsLlm usage normalizer
@agentcosts/llm-usage-normalizer
Convert Anthropic and OpenAI usage exports, or your own app's token logs, into one usage schema with customers, tasks, cached and uncached tokens, and cost from a price table you edit. Fixes the classic mistake of counting cached tokens twice, and lists unmapped keys and unpriced models.
- Activity
- New: be the first to try it
- Installs
- 0 installs
- Likes
- 0 likes
- Clones
- 0 clones
- 1---
- 2name: llm-usage-normalizer
- 3description: Convert LLM usage exports from Anthropic and OpenAI (usage reports, admin API dumps, or your own app's token logs) into one clean usage CSV with customer, task, uncached and cached tokens, and cost from an editable price table. Maps API keys, projects, or workspaces to customers and lists anything it could not map or price. Use when the user has a provider usage export, wants to combine providers, asks "what did each customer cost us in tokens", or needs input for the agent-cost-dashboard, gross-margin-by-customer, usage-pricing-simulator, or consumption-guardrails kits.
- 4license: MIT
- 5---
- 7# LLM usage normalizer
- 9Every provider counts tokens a little differently. The big trap: OpenAI-style exports count cached tokens inside `input_tokens`, while Anthropic reports uncached input separately. Add them up naively and you double count. This kit converts everything to one schema where `input_tokens` always means uncached input, then prices it from a table you control.
- 11## Output schema
- 13```csv
- 14timestamp,customer,task,run_id,provider,model,input_tokens,output_tokens,cache_read_tokens,cache_write_tokens,cost_usd,outcome
- 15```
- 17## Expected input columns
- 19The script finds columns by name (first match wins). Rename columns that differ, or pass `--source`.
- 21| Source | Time | Model | Input | Cached input | Cache writes | Output | Customer key |
- 22| --- | --- | --- | --- | --- | --- | --- | --- |
- 23| Anthropic usage report (flattened) | `starting_at` | `model` | `uncached_input_tokens` (uncached) | `cache_read_input_tokens` | `cache_creation.ephemeral_5m_input_tokens`, `cache_creation.ephemeral_1h_input_tokens` | `output_tokens` | `api_key_id` or `workspace_id` |
- 24| Anthropic API responses logged by your app | `created_at` or `timestamp` | `model` | `input_tokens` (uncached) | `cache_read_input_tokens` | `cache_creation_input_tokens` | `output_tokens` | your own `customer` column |
- 25| OpenAI usage export (completions) | `start_time` (unix seconds) | `model` | `input_tokens` (includes cached) | `input_cached_tokens` | none | `output_tokens` | `project_id`, `api_key_id`, or `user_id` |
- 26| OpenAI API responses logged by your app | `created_at` | `model` | `prompt_tokens` or `input_tokens` (includes cached) | `prompt_tokens_details.cached_tokens` or `input_tokens_details.cached_tokens` | none | `completion_tokens` or `output_tokens` | your own `customer` column |
- 28Also read when present: `customer`, `task` (or `workflow`, `feature`, `agent`), `run_id` (or `trace_id`, `session_id`), `outcome` (or `status`), `batch` or `service_tier` (batch rows get `--batch-discount`, default 0.5), and `cost_usd` (kept as is).
- 30Export formats change. If a column is missing or named differently, show the user the header and map it by hand rather than guessing.
- 32## Inputs
- 341. One or more exports in `data/`.
- 352. **Price table** `data/llm-prices.csv`. If missing, copy `examples/prices.csv` there. Columns: `provider,model,input_per_mtok,output_per_mtok,cache_read_per_mtok,cache_write_per_mtok,checked_on,notes`, plus optional `cache_write_1h_per_mtok` (defaults to 2x input). Model names match exactly or by prefix, so `gpt-5-mini` prices `gpt-5-mini-2025-08-07`.
- 363. **Customer map** `data/customer-map.csv` (`key,customer,task`) when exports only know API keys, projects, or workspaces. One key per customer is the cleanest setup; suggest it if they share keys.
- 38## Steps
- 401. **Check prices first.** Tell the user plainly: the example prices are a starting point and must be checked against each provider's current pricing page. Ask them to confirm or update `data/llm-prices.csv` and set `checked_on`. The script warns when prices were never checked or are older than 60 days.
- 412. Run:
- 43 ```bash
- 44 node scripts/normalize.mjs data/anthropic-usage.csv data/openai-usage.csv --prices data/llm-prices.csv --customer-map data/customer-map.csv --out out/usage-normalized
- 45 ```
- 473. Read `out/usage-normalized-report.md`. Fix, then rerun:
- 48 - **Keys with no customer:** ask who each key belongs to and add it to `data/customer-map.csv` (internal, staging, and eval keys map to `internal`).
- 49 - **Models with no price:** add a row to the price table after the user confirms the price.
- 504. Hand off `out/usage-normalized.csv` to the next kit (agent-cost-dashboard, gross-margin-by-customer, usage-pricing-simulator, consumption-guardrails).
- 52## Limits to explain
- 54- Provider reports are usually daily totals per model and key, not single calls. They have no `run_id` or `outcome`, so cost per successful outcome needs your app's own logs.
- 55- If the provider bill and this total differ by more than a few percent, look for: wrong prices, batch or priority tiers, long-context pricing, tool or search fees, credits, and taxes. The script only prices tokens.
- 57## Guardrails
- 59- Never put API keys or secrets in any file. Key IDs and names from usage reports are fine; key values are not. If an export contains full key values, stop and ask the user to remove that column.
- 60- This kit reads files only. It never calls a provider API.
- 61- Prices are the user's responsibility to confirm. Always say so next to any cost total.