Enrichment waterfall planner

Enrichment waterfall planner

A waterfall asks provider 1, then sends the misses to provider 2, and so on. Order changes cost a lot: a cheap per-lookup provider first, an expensive per-match provider later, is often half the price for the same fill. This kit does that math for every order.

When to use

• Setting up or reworking an enrichment flow for one data point (work email, mobile, direct dial, firmographics).
• After a bake-off: "we ran 200 records through four providers, what now?"
• Budgeting: "what will 10,000 records cost at 80% fill?"
• Not for choosing vendors on features, contracts, or compliance. Mention those as open questions; do not rank vendors yourself.

Inputs

1. Providers in data/providers.json (see the kit's examples/providers.json). Use neutral names if the user prefers (provider-a). Per provider:

   • cost per call, and charge: per_lookup (pay for every query) or per_match (pay only for a found value).
   • match_rate (0 to 1): share of records it finds a value for. Not needed with a sample.
   • accuracy (0 to 1): share of found values that are correct (for example the share that pass email verification). Default 1.
2. Run settings in the same file: records, optional min_fill_rate (usable fill target, 0 to 1), optional max_cost_per_record (flags steps that cost more per extra record), optional verify.cost (turns on verification: values that fail continue down the waterfall).
3. Sample run (strongly recommended) in data/sample.csv: one row per test record, one column per provider name, non-empty when that provider returned a value. Overlap between providers is measured, not assumed. Without a sample the planner assumes providers miss independently, which usually overstates the value of extra providers.

Ask where the numbers came from. Vendor-claimed match rates are usually higher than what a test on the user's own records shows.

Steps

1. Build data/providers.json with the user. Confirm prices and charge models.

2. Run:

   node scripts/waterfall.mjs data/providers.json --sample data/sample.csv --out out --name waterfall

   Leave out --sample if there is no test run.

3. Read out/waterfall.json. Explain:

   • The recommended order, its usable fill, total cost, and cost per usable record.
   • Each step's marginal cost per extra usable record. The last steps are usually the most expensive; say if a step costs more than a record is worth to the user.
   • best_by_depth: what 1, 2, 3 providers get you. This is the "is another provider worth it" answer.
4. If the sample is small (under 200 records), say the rates are rough and suggest a bigger test before signing a contract.
5. Write the plan into out/waterfall-plan.md: order, stop rules, verification step, expected monthly volume and cost, and the open questions.

Output to the user

**Recommended: provider-c > provider-a > provider-b** for verified work email.
77.1% usable fill on 10,000 records, about 1,592 total, 0.206 per usable record.

| Step | Provider | Lookups | Usable | Extra usable cost |
| ---: | --- | ---: | ---: | ---: |
| 1 | provider-c | 10,000 | 3,200 | 0.099 |
| 2 | provider-a | 6,800 | 2,208 | 0.251 |
| 3 | provider-b | 4,592 | 2,305 | 0.313 |

Adding provider-d gets 89.3% fill for about 621 more (about 0.51 per extra usable record). Worth it only if a verified email is worth more than that to you.

Guardrails

• Vendor-neutral. Do not recommend, rank, or name providers the user did not name. Do not claim partnerships or endorsements.
• Numbers are estimates. Always show where the rates came from (estimates or a sample of N).
• Remind the user to check each provider's terms and the privacy rules that apply to their contacts (for example consent and opt-out handling) before enriching personal data.
• Never call provider APIs from this kit. It plans; the user's tools run the waterfall.

Enrichment waterfall planner

Works out the best order for your enrichment providers and where to stop. It tries every order, models per-lookup vs per-match pricing, accuracy, and verification, and tells you the cost per usable record and what each extra provider really adds.

Who it is for

GTM engineers, RevOps, and growth teams enriching contacts or accounts with several data providers, who want to hit a fill target without overpaying. Vendor-neutral: you name the providers.

Example

Input: examples/providers.json (four providers, verified work email, 10,000 records, 70% target) and examples/sample.csv (a 10-record test run).

Output (out/waterfall.md):

Recommended order: provider-c > provider-a > provider-b: 77.1% usable fill,
total 1591.9, 0.206 per usable record.
Best by number of providers: 1 = 58.2%, 2 = 78.9%, 3 = 88.4%, 4 = 89.3% fill.

With per-step lookups, found, usable, cost, and the marginal cost of each extra usable record.

What it can access

• Runs node scripts/waterfall.mjs (Node built-ins only, no network, no provider APIs).
• Reads data/ (plus the kit's own examples/), writes data/ and out/.

Setup

None. Put your providers' prices and rates in data/providers.json, ideally with a sample test run in data/sample.csv.