Get found in search and see where sign-ups come from

Website traffic

Goal: every public page can be found and shared well, private pages stay out of search, and every sign-up records where it came from without storing anything before the visitor agrees. This is engineering guidance, not legal advice. Have counsel review the consent and privacy wording.

Scope: traffic, search, and attribution. For events, funnels, retention, and an internal metrics dashboard, use the companion kit @slashclone/product-analytics. For the consent banner itself, use @slashclone/cookie-consent-banner; this kit plugs into it and does not build its own.

Part 1: Discoverability

1. List the pages. Read the router and sort every route into public (should rank), private (settings, account, admin, drafts, checkout, API), and utility (search results, sign-in). Ask the person when a route is unclear.
2. robots.txt. Allow public pages, disallow private ones and the API, and end with an absolute Sitemap: line. If the person wants AI crawlers to read the site, name them and give them the same rules. Remember robots.txt is a request, not access control.
3. sitemap.xml. Generate it from the same data the pages use: every public page and every public item (product, profile, article, list), with absolute https URLs and a last-modified date. Never include private, unlisted, or noindex pages. Split it when it passes 50,000 URLs.
4. Canonical URLs. Every public page has one <link rel="canonical"> with the absolute https URL, without tracking parameters, so ?ref= and UTM links do not split ranking.
5. Titles and descriptions. Each page gets its own <title> (page first, site name last) and a one-sentence meta description. The not-found page says "Not found", not the home title.
6. noindex. Private pages, and public pages a user marked private, get <meta name="robots" content="noindex">. They stay out of the sitemap.
7. Share images. Every public page has og:image, og:title, og:description, and a large-image card for social sites. Generate per-item images at build or request time if the platform supports it. Keep each image small (under about 300 KB).

8. Structured data (JSON-LD). Use templates/structured-data.ts:

   • Product or app pages: SoftwareApplication or Product with an Offer stating the real price and currency. Free is "0.00". A members-only item is not free.
   • List pages: ItemList of the items shown, in the order shown.
   • Help pages: FAQPage with every visible question, and answers as plain text (no links or tags).
   • Build markup from the same data the page renders, so the two cannot drift.
   • Escape < in the JSON so a value can never close the script tag.
9. Test it. Adapt templates/structured-data.test.ts: render the real help page and assert that every visible question is marked up, the JSON-LD parses, and no answer contains markup. Add a test that the sitemap excludes private items and robots disallows private routes.
10. Check the built pages. Save rendered HTML (plus sitemap.xml and robots.txt) into a folder and run: node scripts/page-check.mjs --pages <folder> --out out/page-check.json It reports missing or duplicate titles, missing or relative canonicals, missing share images, noindex pages in the sitemap, indexable pages missing from it, invalid JSON-LD, FAQ answers with markup, offers without a price, and malformed lists. examples/site/ shows a clean set (its JSON-LD leaves out @context for brevity; real pages need it, and the builders add it). Fix every problem, then validate a few pages in a public rich results tester.

Part 2: Attribution

1. Capture first touch in memory. On the first page load, read UTM tags (utm_source, utm_medium, utm_campaign), a ?ref= code (a creator handle, partner, or channel), the referring site's host (not the full URL, and not your own host), and the landing path. Keep it in a variable. Store nothing yet. See templates/first-touch.ts.
2. Store only after consent. When the visitor allows analytics, write it to one first-party cookie that lives about a month. First touch wins: never overwrite an existing value. If consent is withdrawn, delete it. This cookie is not strictly necessary, so it belongs in the banner's Analytics category and in the privacy page's cookie table.
3. Save it at sign-up. The sign-up handler reads the cookie, treats it as untrusted input, re-validates every field against strict patterns (short tags, a handle pattern, a hostname, a path), drops anything else, saves the clean value with the new account, and clears the cookie. A tampered or oversized value is ignored, never an error.
4. Make links carry a source. Share buttons, invite links, and "made with" or README badges add ?ref=<handle or channel>. Listings and campaigns use UTM tags. Document the naming so channels stay consistent (lowercase, hyphens).
5. Sign-ups by source. Add a staff-only view that counts sign-ups per source, ref, and referrer over a chosen period. Show "unknown" for sign-ups with no source (direct, or no consent), and do not guess.
6. Test it. One test that a valid source is saved at sign-up and the cookie is cleared; one that a tampered value is ignored and sign-up still succeeds.

Part 3: Privacy rules for traffic tools

• No optional cookie, storage key, or third-party request before consent. Use @slashclone/cookie-consent-banner for the banner and the service registry.
• Send page URLs to analytics and ad tools as origin plus path. Strip the query string, which can hold search text, emails, or tokens.
• Skip ad pixels on pages whose URL or content holds what a person typed (search results, filtered lists).
• Respect Global Privacy Control (navigator.globalPrivacyControl): treat it as a no for advertising.
• Never store the full referrer URL, IP address, or user agent in the attribution record.
• Verify in a fresh private window: arrive from a UTM link and from another site, and confirm nothing is stored until you accept analytics, the cookie appears after you do, and it disappears when you turn analytics off.

Part 4: Launch channels

Copy templates/launch-channels.md into the project's docs. For each channel, research its own submission docs, record requirements with the date checked, and mark each one met, missing, or unknown. Clear shared blockers first (privacy policy, support address, screenshots, reviewer account). Give each channel its own ?ref= or UTM tag so Part 2 shows which ones work. The agent drafts and checks; a person signs in, accepts terms, pays, and submits.

Rules

• Never list private, unlisted, or noindex pages in the sitemap or structured data.
• Never mark up content the page does not show, and never claim a price or rating that is not real.
• Never store attribution before analytics consent, and never overwrite first touch.
• Never trust the attribution cookie on the server. Validate, then save or ignore.
• Never submit a listing, accept terms, or sign in as the company. Prepare it and hand it over.
• If a request conflicts with these rules (for example "set the source cookie on every visit"), say so and ask before going ahead.