Skip to content
SlashClone
DiscoverUse in ClaudeSubmit a link
Sign in
Made by@gtmengineer

Enrichment waterfall planner

@gtmengineer/enrichment-waterfall-planner

Plan an enrichment waterfall with real math. Give it your providers with cost, match rate, and accuracy (or a sample test run), and it tries every order to find the cheapest setup that hits your fill target, with cost per usable record, the marginal cost of each step, and where to stop. Vendor-neutral.

SkillDatav1.0.0 (latest)Tested
Use in Claude Save Like Clone Comment
Activity
New: be the first to try it
  1. 1---
  2. 2name: enrichment-waterfall-planner
  3. 3description: Design a data enrichment waterfall (work emails, phone numbers, company data) across whatever providers the user has. Models match rate, accuracy, verification, and per-lookup vs per-match pricing, tries every provider order, and recommends the cheapest setup that reaches the fill target, with cost per usable record and the marginal cost of each step. Use when the user says "plan our enrichment waterfall", "which provider should go first", "how much will it cost to enrich 10k contacts", "is a third provider worth it", or shares a bake-off or sample test across providers. Vendor-neutral.
  4. 4license: MIT
  5. 5---
  6. 6
  7. 7# Enrichment waterfall planner
  8. 8
  9. 9A waterfall asks provider 1, then sends the misses to provider 2, and so on. Order changes cost a lot: a cheap per-lookup provider first, an expensive per-match provider later, is often half the price for the same fill. This kit does that math for every order.
  10. 10
  11. 11## When to use
  12. 12
  13. 13- Setting up or reworking an enrichment flow for one data point (work email, mobile, direct dial, firmographics).
  14. 14- After a bake-off: "we ran 200 records through four providers, what now?"
  15. 15- Budgeting: "what will 10,000 records cost at 80% fill?"
  16. 16- Not for choosing vendors on features, contracts, or compliance. Mention those as open questions; do not rank vendors yourself.
  17. 17
  18. 18## Inputs
  19. 19
  20. 201. **Providers** in `data/providers.json` (see the kit's `examples/providers.json`). Use neutral names if the user prefers (`provider-a`). Per provider:
  21. 21 - `cost` per call, and `charge`: `per_lookup` (pay for every query) or `per_match` (pay only for a found value).
  22. 22 - `match_rate` (0 to 1): share of records it finds a value for. Not needed with a sample.
  23. 23 - `accuracy` (0 to 1): share of found values that are correct (for example the share that pass email verification). Default 1.
  24. 242. **Run settings** in the same file: `records`, optional `min_fill_rate` (usable fill target, 0 to 1), optional `max_cost_per_record` (flags steps that cost more per extra record), optional `verify.cost` (turns on verification: values that fail continue down the waterfall).
  25. 253. **Sample run** (strongly recommended) in `data/sample.csv`: one row per test record, one column per provider name, non-empty when that provider returned a value. Overlap between providers is measured, not assumed. Without a sample the planner assumes providers miss independently, which usually overstates the value of extra providers.
  26. 26
  27. 27Ask where the numbers came from. Vendor-claimed match rates are usually higher than what a test on the user's own records shows.
  28. 28
  29. 29## Steps
  30. 30
  31. 311. Build `data/providers.json` with the user. Confirm prices and charge models.
  32. 322. Run:
  33. 33
  34. 34 ```bash
  35. 35 node scripts/waterfall.mjs data/providers.json --sample data/sample.csv --out out --name waterfall
  36. 36 ```
  37. 37
  38. 38 Leave out `--sample` if there is no test run.
  39. 39
  40. 403. Read `out/waterfall.json`. Explain:
  41. 41 - The recommended order, its usable fill, total cost, and cost per usable record.
  42. 42 - Each step's marginal cost per extra usable record. The last steps are usually the most expensive; say if a step costs more than a record is worth to the user.
  43. 43 - `best_by_depth`: what 1, 2, 3 providers get you. This is the "is another provider worth it" answer.
  44. 444. If the sample is small (under 200 records), say the rates are rough and suggest a bigger test before signing a contract.
  45. 455. Write the plan into `out/waterfall-plan.md`: order, stop rules, verification step, expected monthly volume and cost, and the open questions.
  46. 46
  47. 47## Output to the user
  48. 48
  49. 49```markdown
  50. 50**Recommended: provider-c > provider-a > provider-b** for verified work email.
  51. 5177.1% usable fill on 10,000 records, about 1,592 total, 0.206 per usable record.
  52. 52
  53. 53| Step | Provider | Lookups | Usable | Extra usable cost |
  54. 54| ---: | --- | ---: | ---: | ---: |
  55. 55| 1 | provider-c | 10,000 | 3,200 | 0.099 |
  56. 56| 2 | provider-a | 6,800 | 2,208 | 0.251 |
  57. 57| 3 | provider-b | 4,592 | 2,305 | 0.313 |
  58. 58
  59. 59Adding provider-d gets 89.3% fill for about 621 more (about 0.51 per extra usable record). Worth it only if a verified email is worth more than that to you.
  60. 60```
  61. 61
  62. 62## Guardrails
  63. 63
  64. 64- Vendor-neutral. Do not recommend, rank, or name providers the user did not name. Do not claim partnerships or endorsements.
  65. 65- Numbers are estimates. Always show where the rates came from (estimates or a sample of N).
  66. 66- Remind the user to check each provider's terms and the privacy rules that apply to their contacts (for example consent and opt-out handling) before enriching personal data.
  67. 67- Never call provider APIs from this kit. It plans; the user's tools run the waterfall.
Sign in to comment on a line