Production Deployment — AWS + Cloudflare (as-built)

The Scanners platform is live in production (since 2026-06-10). This page is the as-built record of that deployment: ECS Fargate, one worker image with the queue chosen per service, Cloudflare in front (no public origin), a private data tier, a stable scan-egress IP, and the guardrails and CI/CD around it.

AWS account	scanners-prod = 429134227608
Region	eu-north-1 (Stockholm) — SCP-locked (below)
Live API base	https://modules.binarynetworks.ai/api/v1 (JSON; OpenAPI/Swagger at /docs)
Scan egress IP	NAT Gateway + Elastic IP 13.50.172.131 (stable; customers allow-list this)

Region note. eu-north-1 is the cheapest EU region and carries every service used here (Fargate, RDS, ElastiCache, ECR, S3, Secrets Manager). EU data residency. The region is enforced by an SCP, not just convention.

Architecture

1. Ingress — Cloudflare Access + Tunnel (no public origin)

The API has no public ALB and no inbound ports. A cloudflared task runs in ECS and makes an outbound connection to Cloudflare; the request path is:

Cloudflare proxied DNS → Cloudflare Access (Zero Trust) → Cloudflare Tunnel (cloudflared, no public origin) → ECS Service Connect → api:8000.

• Edge identity — Access service token. A Cloudflare Access service token gates the hostname. Clients send the CF-Access-Client-Id and CF-Access-Client-Secret headers; the Access policy decision is non_identity (Service Auth), which is what lets non-interactive API clients through. (A plain allow policy would bounce API clients to the 403 login page — see Gotchas.)
• Service Connect, not a load balancer, routes the tunnel to the private api service on the scanners.local namespace at api:8000.
• App-layer auth (the 3-key X-Master-Key / X-Admin-Key / X-Org-Key) still applies on top — defense in depth.
• Cloudflare also gives WAF, DDoS protection, and edge rate limiting.

The tunnel token is stored in Secrets Manager (cf-tunnel-token) and injected at task startup — never baked into an image.

2. Network

• VPC 10.0.0.0/16, 2 AZs, public + private subnets.
• Single NAT Gateway (with the EIP above) for egress.
• S3 gateway endpoint + ECR / Secrets Manager interface endpoints so image pulls and secret fetches stay on the AWS network.

3. Compute — ECS Fargate, one image per role, queue by command

ECS Fargate cluster scanners, Service Connect namespace scanners.local. Two ECR repositories: scanners-api (lean) and scanners-worker (Project Discovery + DAST toolchain). There is one worker image — the queue is chosen by the ECS container command, not by building separate worker images.

Service	Image	Command / role	Scaling
api	scanners-api (lean)	the REST API at api:8000	autoscale 1–4 on CPU
easm-worker	scanners-worker	-Q easm	n/a
dast-worker	scanners-worker	-Q dast	n/a
cloudflared	cloudflared	the tunnel	n/a

• Deployment circuit breaker is enabled on the services: a failed rollout auto-rolls-back to the last good task definition.
• Per-module isolation is real — EASM and DAST are separate ECS services with their own task definitions, queues, and metrics; a DAST tool problem cannot affect EASM.

4. Scan egress — stable IP

Workers run in the private subnets behind the NAT Gateway + Elastic IP 13.50.172.131, so all scans leave from one stable, known IP. This is published as the scanner's source IP so customers can allow-list it and attribution stays clean.

5. Data tier

• RDS PostgreSQL 16, db.t4g.small, single-AZ (Multi-AZ is flippable later with no app change), private, encrypted, 7-day PITR. Security group allows only the ECS task SGs.
• ElastiCache Redis 7.1, cache.t4g.small, private — the Celery broker / result backend.

6. Secrets + self-bootstrapping schema

• AWS Secrets Manager holds: db-password, master-api-key, database-url, celery-broker-url, celery-result-backend, cf-tunnel-token, and optional enrichment keys. All are injected at task startup — never baked into images or committed to the repo.
• Schema bootstraps itself. On boot the API's FastAPI lifespan (ensure_shared_schema) self-creates the shared schema; tenant schemas are created on org creation. There is no separate migration step to run before first start.

7. Logging — S3 via FireLens (not CloudWatch Logs)

• Container / stream logs → S3 via the ECS FireLens (Fluent Bit) log driver on each service, written under per-module prefixes in the binary-scanners-logs bucket. Logs do not go to CloudWatch Logs.
• Per-org activity_log stays in RDS — the structured, low-volume business audit the UI queries (GET /org/activity). RDS is right for this.

Example ECS FireLens block (per service):

"logConfiguration": {
  "logDriver": "awsfirelens",
  "options": { "Name": "s3", "region": "eu-north-1",
               "bucket": "binary-scanners-logs", "total_file_size": "50M",
               "s3_key_format": "/$TAG/%Y/%m/%d/%H_%M_%S" }
}

8. Guardrails

• SCP region-lock. A Service Control Policy locks scanners-prod to eu-north-1 (plus global services) and denies leaving the org, toggling enabled regions, and creating root access keys.
• Centralized Root Access is enabled org-wide: the member account's root user has no credentials and sign-in / recovery is disabled.
• Budget. A $300/mo AWS Budget alerts at 50 / 80 / 100% plus a forecast alert.

9. CI/CD — GitHub Actions + OIDC

• No static AWS keys in GitHub — GitHub Actions assumes a role via OIDC.
• On PR / push: ruff + pytest (with Postgres + Redis service containers).
• On push to main: build and push both images to ECR, tagged with the commit SHA and latest.
• Deploy is manual via workflow_dispatch — a free manual-approval gate. (Environment required-reviewers on a private repo would need GitHub Team, so workflow_dispatch is the gate today.) The deploy job registers a new task definition with the SHA image, updates the services, and waits for steady state; the deployment circuit breaker auto-rolls-back on failure.

10. Monitoring (detection built; alert wiring in progress)

Detection is in place: 11 CloudWatch alarms — ECS task-down (via EventBridge), RDS CPU / storage / connections, and Redis CPU / memory — plus an SNS topic scanners-alerts and an RDS event subscription. The SNS → email / Slack destination is still being wired (a 4-layer plan in progress); the alarms and topic exist, the final notification hop does not yet.

Gotchas (and fixes)

These bit us during go-live; recorded so they don't bite again.

1. GitHub OIDC trust must allow two subjects. The trust policy needs both repo:IONTWRKS/scanners:ref:refs/heads/main and repo:IONTWRKS/scanners:environment:production — environment-scoped deploy jobs present a different OIDC sub.
2. ecs:DescribeTaskDefinition has no resource-level ARNs. It must be granted with Resource="*" — scoping it to a task-def ARN fails.
3. Access service-token policy must be non_identity (Service Auth), not allow. With allow, API clients hit the 403 login page instead of being authenticated by the token.
4. cloudflared may need a forced redeploy. If cloudflared started before api was healthy, its Service Connect route is stale and the API returns HTTP 502. Force-redeploy cloudflared to refresh the route — note deploy.yml does not redeploy cloudflared automatically.
5. terraform state show redacts the service-token secret. It hides the Access service-token client_secret; use terraform state pull to read it.

Notes

• The same surface serves future modules: add a <module>-worker service running the shared scanners-worker image with -Q <module>, register its scanners, and the API routes to it automatically.
• naabu caveat: SYN port scanning needs CAP_NET_RAW, which Fargate does not grant. Use naabu connect-scan on Fargate, or run the worker on an EC2-backed ECS capacity provider with NET_RAW if you need SYN scans.
• Multi-AZ on RDS is a flip-the-switch change later; the single-AZ instance is the current cost-conscious baseline.