A 51,000-page directory release with a private publishing control plane
Immutable Rust releases replaced the Laravel public directory; a separate private control plane handles fail-closed sources, duplicates, media, review, and controlled publication.
51,098 pages · 409,495 route outcomes · 321,904 redirects · 4 source adapters
What needed to change
The old application mixed public pages, legacy data, PHP, and a live database on the request path. After removing that runtime, new source intake still needed a safe private path that could not silently put unreviewed records back into public output.
The constraint shaping the build
Preserve the large URL history, keep source and media work private, make duplicate decisions durable, and force every accepted change through a complete validated release.
How the pieces connect
- Legacy data and controlled sources
- Fail-closed adapters
- Duplicate and qualification review
- Restart-safe media queue
- Immutable Rust build and validation
- Blue/green activation
See whether this starting point fits your workflow
The matching system explains who it is for, what a first release includes, and the rough cost. Supporting links show the implementation evidence.
How I built it
- Use the legacy database as migration input, not as a permanent dependency of the new public runtime.
- Generate one authoritative manifest covering live pages, redirects, removals, canonicals, and assets.
- Make AdultSearch the only approved adapter by default; ListCrawler, OnDate, and RubMaps fail closed without explicit configuration.
- Suppress exact cross-source duplicates into a durable ledger while treating similarity as a review hint rather than an automatic merge.
- Bound media fetch and validation, create JPEG and WebP derivatives, verify B2 and Bunny delivery, and make backfill restart-safe.
- Keep quota-bound automatic publishing off by default and require the full versioned build, validation, activation, and rollback path even when enabled.
What I verified
- 51,098 generated public pages
- 409,495 classified route outcomes
- 321,904 redirects
- Four implemented source adapters with three disabled by default
- Private Review, Listings, Sources, Queues, Publishing, and Settings routes
- Durable duplicate ledger and restart-safe media backfill
Current state: The page and route figures belong to the observed public release. In the newer private control plane, only one of four source adapters is approved by default and automatic publishing is disabled by default. It is not multi-tenant and does not add user accounts, payments, or messaging.