Topology and Environments

SterlingIndex is deployed as a split system: independent frontend apps at the edge, and a shared backend runtime that owns data access, auth sessions, background work, and runtime dependencies.

The checked-in topology is defined by two repo-visible layers:

• infra/docker-compose.local.yml for the current monorepo's local backend and stateful services
• infra/vps/* for the current production-oriented single-host backend baseline

The repo also keeps a Kubernetes reference layer:

• infra/k8s/* for a heavier future multi-node backend shape

Repo-visible secret management now has a third layer:

• INFISICAL.md for the recommended Infisical project layout, path mapping, local CLI workflow, and Vercel/Cloudflare sync model

Topology Summary

Local development
browser
  |- apps/www (Next.js, :3000)
  |- apps/dashboard (Vite, :8080)
  |- apps/admin-desktop (Electron admin renderer :3001, local API :13000)
  |- apps/admin-mcp (stdio MCP server, local API :13000 when started)
  |- apps/legacy-admin (legacy Vite reference, :3001)
  |- apps/docs (Next.js, :3001 when run separately)
  `- services/platform-api API (:8000)
          |- Postgres
          |- Redis
          |- MinIO
          |- Ollama
          |- mock DROP API
          |- Celery worker (`scrape-cloud,validation-publish,compliance`)
          `- scheduler tick (`python -m rosterscraper.scheduler_tick`)

optional macOS scrape node
  `- Celery worker (`scrape-local`) against cloud/local Postgres, Redis, and R2-compatible storage

Production baseline
www.sterlingindex.com        dashboard.sterlingindex.com
docs.sterlingindex.com       active admin: macOS Electron
             \                 |                 /
              \                |                /
                 api.sterlingindex.com
                          |
          Cloudflare proxy and origin cert
                          |
      Vercel / Cloudflare Pages / Contabo VPS
                          |
       nginx + Docker Compose on the VPS
                          |
        local Postgres / Redis / R2 / systemd timer
                          |
          optional macOS scrape-local worker over cloud state

Local Development Topology

The root workspace script in package.json starts four processes:

• pnpm dev:backend -> docker compose -f infra/docker-compose.local.yml --env-file infra/.env.local.example up --build
• pnpm dev:www -> apps/www
• pnpm dev:dashboard -> apps/dashboard
• pnpm dev:admin -> apps/admin-desktop

The docs app is intentionally separate and runs with pnpm dev:docs.

The workspace also exposes Infisical-backed equivalents:

• pnpm dev:infisical for the main local stack
• pnpm dev:admin:infisical for the Electron admin with the /apps/admin-desktop secret path
• pnpm dev:docs:infisical for the docs app alone

The old hosted admin is intentionally outside the active admin path. Use pnpm dev:legacy-admin, pnpm build:legacy-admin, or pnpm test:legacy-admin only when you are checking the retained reference app.

The MCP admin server is also outside the default root pnpm dev script. Use pnpm --dir apps/admin-mcp build and configure an MCP client to launch apps/admin-mcp/dist/index.js, or use pnpm --dir apps/admin-mcp start for local development.

Compose-backed services

Optional local scrape node

Hybrid staging and production can attach a separate macOS worker outside Compose:

• command baseline: pnpm --dir services/platform-api dev:worker:local
• queue contract: consumes scrape-local only
• state contract: points at the shared DATABASE_URL, REDIS_URL, OBJECT_STORAGE_*, and ATTORNEY_MEDIA_*
• model/browser contract: can use local OLLAMA_HOST, LMSTUDIO_BASE_URL, and native macOS browser automation without forcing those dependencies onto the hosted API

This worker is optional capacity, not required infrastructure. If it is offline, the hosted platform can continue routing ad hoc runs to scrape-cloud.

Frontends outside Compose

Local environment contract

infra/.env.local.example is the checked-in coordination point for the workspace-level local stack. It defines:

• stateful services: POSTGRES_*, REDIS_PORT, MINIO_PORT, MINIO_CONSOLE_PORT
• backend runtime URLs: DATABASE_URL, INTEL_DATABASE_URL, CORE_DATABASE_URL, REDIS_URL, OLLAMA_HOST, LMSTUDIO_BASE_URL
• orchestration mode and scrape targeting: ORCHESTRATION_EXECUTION_MODE, SCRAPE_EXECUTION_TARGET_DEFAULT
• worker identity and queue advertisement: SCRAPE_NODE_ROLE, SCRAPE_NODE_LABEL, SCRAPE_NODE_PLATFORM, SCRAPE_NODE_ADVERTISED_QUEUES
• public app URLs consumed by the backend: APP_PUBLIC_WWW_URL, APP_PUBLIC_DASHBOARD_URL, APP_PUBLIC_ADMIN_URL, APP_PUBLIC_API_URL
• frontend runtime URLs: VITE_PLATFORM_API_URL, VITE_CONTROL_PLANE_URL, NEXT_PUBLIC_DASHBOARD_URL, NEXT_PUBLIC_DOCS_BASE_URL
• auth/session defaults: AUTH_PROVIDER, WORKOS_*, CORS_ALLOWED_ORIGINS, SESSION_COOKIE_NAME, SESSION_COOKIE_DOMAIN, SESSION_COOKIE_SECURE
• observability inputs: APP_RELEASE, MONITORING_SENTRY_DSN, MONITORING_SENTRY_TRACES_SAMPLE_RATE, NEXT_PUBLIC_APP_ENV, NEXT_PUBLIC_APP_RELEASE, NEXT_PUBLIC_SENTRY_DSN, NEXT_PUBLIC_SENTRY_TRACES_SAMPLE_RATE, NEXT_PUBLIC_POSTHOG_KEY, NEXT_PUBLIC_POSTHOG_HOST, VITE_APP_ENV, VITE_APP_RELEASE, VITE_SENTRY_DSN, VITE_SENTRY_TRACES_SAMPLE_RATE, VITE_SENTRY_REPLAY_SESSION_SAMPLE_RATE, VITE_SENTRY_REPLAY_ON_ERROR_SAMPLE_RATE, VITE_POSTHOG_KEY, VITE_POSTHOG_HOST
• security, storage, and media settings: OBJECT_STORAGE_*, ATTORNEY_MEDIA_*, VITE_TURNSTILE_SITE_KEY, VITE_OPENFREEMAP_COMPACT_STYLE_URL, VITE_OPENFREEMAP_IMMERSIVE_STYLE_URL, SECURITY_TURNSTILE_SECRET_KEY, PRODUCT_DEMO_SEED_ENABLED

The docs app keeps its own .env.example because it publishes docs-site metadata from NEXT_PUBLIC_DOCS_BASE_URL and the shared dashboard-login handoff from NEXT_PUBLIC_DASHBOARD_URL.

Secrets source of truth

The repo-level recommendation is now:

• use Infisical as the source of truth for local and hosted secrets,
• sync apps/www and apps/docs public env vars into Vercel,
• sync apps/dashboard env vars into Cloudflare Pages,
• keep active admin env values under the /apps/admin-desktop Infisical path or import them through the desktop app's env-import workflow,
• import Admin MCP env bundles through sterling_runtime_import_env_bundle; the cache is encrypted under ~/Library/Application Support/Sterling Admin MCP/ unless plaintext cache is explicitly enabled for tests,
• sync apps/legacy-admin to Cloudflare Pages only if you intentionally deploy the reference web admin,
• use Infisical machine identities for the API, worker, and scheduler where possible.

The checked-in operating guide for that model is INFISICAL.md.

Production-oriented Topology

The checked-in deployment runbook now treats infra/vps as the primary production backend path and keeps the frontend apps as independent deployments:

• apps/www on Vercel
• apps/docs on Vercel for docs.sterlingindex.com
• apps/dashboard on Cloudflare Pages
• apps/admin-desktop as the packaged Electron control room for operators
• apps/admin-mcp as a local stdio MCP control surface for agent-assisted operators
• apps/legacy-admin retained as a reference Vite app, not a default deploy target
• services/platform-api plus worker, scheduler, Postgres, and Redis on a single Contabo VPS

VPS baseline

Operationally, that means the repo-visible production backend is now:

• Cloudflare proxy in front of api.sterlingindex.com
• Nginx on the VPS listening on 80/443
• FastAPI, worker, scheduler tick loop, Postgres, and Redis on an internal Docker network
• Cloudflare R2 retained for object storage and media
• Infisical machine identity used to inject backend runtime secrets

Kubernetes reference baseline

The Kubernetes manifests remain useful as a future multi-node reference, but they are no longer the default deployment story for the current production path.

Production domain map

Neither the VPS bundle nor the Kubernetes reference includes manifests for the frontend hosts themselves, so the repo-visible production picture remains "independent frontend deployments pointed at one shared API origin."

The repo now also includes infra/cloudflare/README.md, which is the checked-in runbook for the intended public edge shape:

• proxy www.sterlingindex.com, dashboard.sterlingindex.com, and api.sterlingindex.com through a single Cloudflare zone
• enable the free-plan baseline for DDoS mitigation, Bot Fight Mode, and targeted WAF/rate-limit rules
• keep the api origin proxied to the Contabo VPS behind Nginx and a Cloudflare Origin Certificate
• keep origin-specific headers and build-time sourcemap behavior in the app repos, because Cloudflare config itself remains an external deployment concern

At the secret-management layer, hosted frontends are expected to receive environment values through Infisical Secret Syncs rather than ad hoc manual editing inside each platform dashboard.

Shared Environment Variables

The environment contract is split by responsibility rather than by app.

Public origin and redirect settings

State, storage, and runtime dependencies

Session and cross-origin assumptions

• Local Compose defaults use SESSION_COOKIE_DOMAIN=localhost.
• The VPS production baseline switches to SESSION_COOKIE_DOMAIN=.sterlingindex.com and SESSION_COOKIE_SECURE=true so one product/auth session can span www, dashboard, and api subdomains. The active Electron admin still uses the control-plane admin session against its configured API origin.
• services/platform-api/src/rosterscraper/config.py checks in localhost defaults, but CORS_ALLOWED_ORIGINS can now be supplied as a comma-separated list or JSON array for production origins such as www, dashboard, and admin.
• In hosted mode, ad hoc discovery and run requests can resolve execution_target=auto to scrape-local when a healthy macOS node is advertising that queue; otherwise they fall back to scrape-cloud. Scheduled jobs stay cloud-oriented by default.

Secret ownership assumptions

• Frontend-only public build vars such as NEXT_PUBLIC_* and VITE_* should live in app-specific Infisical paths.
• Shared backend secrets such as WORKOS_*, SESSION_COOKIE_*, DATABASE_URL, REDIS_URL, and OBJECT_STORAGE_* should live in shared runtime paths and be imported into API/worker/scheduler scopes.
• Build-only sourcemap credentials such as SENTRY_AUTH_TOKEN, SENTRY_ORG, and project-name overrides should stay separate from runtime secrets so frontend deploys can read them without widening backend access.

Verification and Operational Notes

• services/platform-api/scripts/run_local_verification.py treats infra/docker-compose.local.yml as the shared dependency stack and exports host-side DATABASE_URL, REDIS_URL, OBJECT_STORAGE_*, and app URL variables before running backend and frontend verification.
• services/platform-api/docs/local-verification.md documents an additional collision-resistant smoke path for the legacy embedded web shell on localhost:13000; that shell is useful for backend verification but is not part of the primary monorepo deployment picture.
• DEPLOY.md is the current repo-level runbook for the split hosted stack, the VPS backend bundle, the scheduler tick loop, and the optional macOS scrape node that writes directly into the shared cloud state.
• INFISICAL.md is the repo-level runbook for the Infisical project shape, path mapping, CLI workflow, and Vercel/Cloudflare Secret Sync setup.
• infra/cloudflare/README.md is the repo-visible runbook for Cloudflare proxy, WAF/rate-limit, and Bot Fight Mode rollout across www, dashboard, and api.
• infra/vps/README.md is the checked-in host runbook for Docker Compose, Nginx, systemd, Cloudflare Origin Certificates, and nightly Postgres backups to R2.
• The docs site itself is verified with the workspace-level scripts: pnpm build:docs, pnpm lint:docs, and pnpm typecheck:docs.