nikom.workIndependent engineering studio

Rust / Analytics / Search visibilityWorking multi-property self-hosted system

Multi-site traffic and search visibility without permanent visitor profiles

A self-hosted system joins stateless multi-site traffic, landing-source attribution, GSC and Bing data, crawler observations, view access, and rollup-first cleanup.

Approximately 3 KB gzipped tracker · 84-operation OpenAPI contract · permanent rollups with safe cleanup

The job

What needed to change

Traffic, first-touch acquisition, search visibility, and AI/search crawlers usually live in separate tools. Raw event storage also grows long after old rows stop supporting day-to-day decisions.

The hard part

The constraint shaping the build

Measure enough across several properties to operate them, preserve permanent history before cleanup, and avoid cookies, browser storage, raw IP retention, or inflated identity and crawler claims.

System flow

How the pieces connect

  1. Approximately 3 KB gzipped stateless tracker
  2. Property-scoped visitor-day
  3. Campaign and landing-source rollups
  4. GSC, Bing, crawler, and index observations
  5. Permanent snapshots
  6. Dry-run and bounded raw cleanup
Matching system

The matching system explains who it is for, what a first release includes, and the rough cost. Supporting links show the implementation evidence.

Private analytics and search visibilitySee the configurable system for first-party traffic, search, crawler, index, retention, and access controls. How to delete raw analytics events without losing historical reportsThe retention, cleanup, access, and identity boundaries. Technical search companion ↗IndexLane owns crawl, indexation, migration, and measurement implementation.
Build decisions

How I built it

  1. Use no cookies or browser storage and do not write raw IP addresses, full user agents, or Client Hints.
  2. Report visitor-days instead of pretending a self-hosted identifier follows one person forever.
  3. Keep permanent daily, search, crawler, index-count, and first-touch landing-source rollups before deleting eligible raw rows.
  4. Make cleanup opt-in at 00:05, expose a dry run, delete in resumable batches, and guard checkpoint and VACUUM.
  5. Scope hashed, revocable view invitations to one property and keep exact-day exports available through the API.
Verification

What I verified

  • Tracker measured 3,066 bytes with GNU gzip's default compression level and header
  • Multi-property traffic, campaign, goal, journey, referrer, and landing-source views
  • Google Search Console, Bing, operator-triggered AParser counts, and user-agent-claimed edge crawler observations
  • Permanent rollups, dry-run cleanup, resumable deletes, and guarded SQLite maintenance
  • 84 method-bearing OpenAPI operations; tracker tests 9/9, Svelte check zero issues, and dashboard build completed

Current state: Visitor-days are estimates, not unique users. Edge crawler identity is user-agent-claimed unless source IPs are validated. AParser index counts are operator-triggered, and exact-day page/source exports are currently API-only.