Architecture
This section describes the production architecture (one region). Environment topology (dev / nonprod / production / Phase 2 IL) is covered separately — see the "Environments" section.
Pattern: Backend-for-Frontend (BFF)
- Frontend — React SPA, pure UI shell. No secrets, no external API calls, no business logic.
- Backend — NestJS BFF. Single gateway for all requests: auth, data fetching, validation, business rules, integration with external services.
- Data flow: React SPA → Cloudflare → AWS ALB → NestJS BFF → PostgreSQL / WorkOS / S3 / external services
Production region architecture (one AWS region)
Internet
│
▼
┌──────────────────────────────────────────┐
│ Cloudflare (single edge layer) │
│ • WAF (OWASP Core Ruleset + custom) │
│ • DDoS L3/4/7 mitigation │
│ • Rate limit (per-IP, per-API-key) │
│ • Bot management │
│ • CDN (300+ PoPs, static SPA assets) │
│ • Proxied DNS (origin IP hidden) │
│ • TLS 1.2+ Full Strict │
└────────────────┬─────────────────────────┘
│ (only Cloudflare IPs +
│ mTLS Authenticated Origin Pulls)
▼
┌──────────────────────────────────────────┐
│ AWS Production Account (set-prod-eu) │
│ ┌───────────────────────────────────┐ │
│ │ VPC (/16, 3 Availability Zones) │ │
│ │ │ │
│ │ Public subnets (1 per AZ) │ │
│ │ └─ ALB │ │
│ │ (SG: inbound = CF IPs only) │ │
│ │ │ │
│ │ Private subnets — compute │ │
│ │ ├─ Fargate: api (NestJS BFF) │ │
│ │ ├─ Fargate: api-workers │ │
│ │ ├─ Fargate: compliance-workers │ │
│ │ └─ Fargate: ai-workers (Phase 2) │ │
│ │ │ │
│ │ Private subnets — data │ │
│ │ ├─ RDS Postgres 16 Multi-AZ │ │
│ │ │ + 1 Read Replica │ │
│ │ │ + RDS Proxy (pooling) │ │
│ │ ├─ ElastiCache Redis Multi-AZ │ │
│ │ │ (sessions, BullMQ queues) │ │
│ │ └─ Keycloak (Phase 2, AWS IL) │ │
│ │ │ │
│ │ NAT Gateway per AZ │ │
│ │ VPC Endpoints (S3, ECR, Secrets │ │
│ │ Manager, KMS, CloudWatch Logs) │ │
│ └───────────────────────────────────┘ │
│ │
│ Managed / shared services │
│ ├─ S3 (KMS-encrypted, all buckets) │
│ │ ├─ Evidence (versioning + Object Lock)│
│ │ ├─ Documents / reports │
│ │ └─ Backups + CloudTrail archive │
│ ├─ Secrets Manager (DB, JWT, API keys) │
│ ├─ KMS (Customer-Managed Keys) │
│ ├─ SSM Parameter Store (non-secret cfg) │
│ ├─ SES (transactional email) │
│ ├─ SNS (internal alerting fanout) │
│ ├─ CloudTrail multi-region trail │
│ │ → S3 with Object Lock │
│ ├─ AWS Config (resource compliance) │
│ ├─ GuardDuty (threat detection) │
│ ├─ VPC Flow Logs │
│ ├─ AWS Backup (cross-service orchestrate) │
│ ├─ IAM Identity Center (SSO + MFA) │
│ └─ CloudWatch (transit; see Observability)│
└──────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────┐
│ External (managed sub-processors) │
│ ├─ WorkOS (auth, EU region) │
│ ├─ Logz.io (observability + SIEM, EU) │
│ └─ Customer-connected systems │
│ (Okta, AWS, GCP, Azure, GitHub, etc.) │
└──────────────────────────────────────────┘
Frontend deployment
- React SPA built as static files → uploaded to private S3 bucket
- Served via
app.{domain}through Cloudflare → S3 (bucket policy restricts to Cloudflare) - Cloudflare CDN caches at 300+ PoPs globally — instant first paint
- Cache invalidation via Cloudflare API on deploy
- No CloudFront — Cloudflare handles all CDN duties
Backend deployment
NestJS BFF + workers run on ECS Fargate. Served via api.{domain} through Cloudflare → ALB → Fargate tasks.
Three Fargate task families (four with Phase 2), each scales independently:
| Task family | Holds | Scales on |
|---|---|---|
api | NestJS BFF | request count + p99 latency |
api-workers | Notifications, reports, exports, audit log persistence, webhooks, tenant lifecycle, maintenance | BullMQ queue depth |
compliance-workers | Evidence ingestion, ingestion of customers' security tool output, compliance computation (rule engine). Runs no scans. | Queue depth, higher CPU/memory profile |
ai-workers (Phase 2) | AI Assistant, embedding generation, RAG over evidence | Queue depth |
Deployment: rolling updates with circuit breaker + auto-rollback on health-check failure. No long-lived credentials in containers — IAM roles per task family.
External system: scanners. No task family above performs scanning. Scanning is a separate live product in AWS account 429134227608, with its own VPC, database and worker fleets, reached by S.E.T as an API client. It is deliberately outside this diagram's trust boundary. It currently has no load balancer and no private endpoint — a Cloudflare tunnel is its only ingress — so the private path from S.E.T to it is unbuilt work, not configuration. Details and constraints (including the 10.0.0.0/16 range S.E.T must not reuse): Environments, "The two scanners accounts".
Network / Edge detail
Cloudflare (single edge layer):
- WAF — OWASP Core Ruleset + custom API rules
- DDoS L3/4/7 mitigation
- Rate limiting per-IP, per-API-key
- Bot management
- TLS 1.2+ Full Strict
- Edge caching at 300+ PoPs
- Proxied DNS — origin IP never exposed
ALB:
- Security group: inbound restricted to Cloudflare IP ranges
- mTLS Authenticated Origin Pulls — cryptographic verification of authentic Cloudflare traffic
- HTTPS termination, path-based routing
- Health-checks Fargate tasks across AZs
AWS Shield Standard — enabled by default, free, covers residual L3/L4 DDoS at the AWS edge.
Intentionally NOT used:
- AWS WAF — redundant given Cloudflare + IP allowlist + mTLS (see Tech Stack Item 9)
- AWS Shield Advanced — $3k/mo+ unnecessary; Cloudflare absorbs DDoS at edge
- CloudFront — Cloudflare handles all CDN duties
Data layer
- PostgreSQL 16 Multi-AZ — primary + standby across AZs (automatic failover)
- 1 Read Replica — analytics + heavy dashboard queries (offloads OLTP)
- RDS Proxy — to be re-evaluated, not assumed. An earlier draft called it "critical for
search_pathswap discipline"; that rationale is void, because isolation now comes from a per-tenant role and grants rather than from swappingsearch_pathon a shared connection. Worse, it may be actively pointless here: AWS lists "UsingSETcommands" among the conditions that cause pinning for RDS for PostgreSQL, and states that "for PostgreSQL setting a variable leads to session pinning" — theSET LOCALexemption is documented for MySQL and MariaDB only, so the transaction-local form gets no PostgreSQL exemption. Pinning holds a client connection to one database connection until the session ends. Since the tenant guard sets bothroleandsearch_pathinside every transaction, every connection through the proxy would pin, removing the multiplexing that is the only reason to run it. It would also need a Secrets Manager secret registered per database user, meaning one per tenant. Decide this when the data layer is built; do not treat it as settled. - Automated backups — 35-day PITR window in production
- Cross-region encrypted backups for catastrophic recovery
- KMS encryption at rest with Customer-Managed Keys (CMKs); opens BYOK premium-tier path
- ElastiCache Redis Multi-AZ — sessions, rate-limit state, query cache, BullMQ queues
- TimescaleDB extension on Postgres for BCP time-series data (uptime, SLA, dependency-graph metrics) — no separate time-series tier needed
- pgvector extension available for Phase 2 AI Assistant RAG embeddings
Async / queue layer
- BullMQ on Redis — primary background-job queue. Richer features than SQS (delayed jobs, priorities, repeating jobs, per-queue rate limiting), and Redis is already in the stack.
- Dead-letter queue for failed jobs — audit trail of what didn't process
- Migration to SQS remains a viable path if BullMQ ops ever become problematic — both are well-understood patterns.
Object storage (S3)
- Evidence bucket — customer-uploaded evidence artifacts. KMS-encrypted with CMK. Versioning + Object Lock for tamper-evidence (regulatory requirement).
- Documents bucket — generated reports, exports, customer downloads.
- Backup bucket — RDS exports + CloudTrail archive. Object Lock for 7-year retention.
- Lifecycle policies — old evidence to Glacier Deep Archive after retention threshold.
- All buckets accessed via VPC Endpoints — no public-internet egress for AWS-service traffic.
Secrets, keys & config
- AWS Secrets Manager — DB credentials, JWT signing keys, external API tokens. Auto-rotation, audited.
- AWS KMS — encryption keys with Customer-Managed Keys (CMKs). Per-environment keys never cross account boundaries. BYOK path for premium tier.
- AWS SSM Parameter Store — non-secret config.
Security & compliance services
- AWS GuardDuty — anomaly detection at AWS layer (compromised credentials, anomalous API calls, crypto-mining). Acts as "EDR for AWS infra"; findings feed Logz.io as primary SIEM.
- AWS Config — resource configuration tracking + drift detection. SOC 2 evidence.
- CloudTrail — multi-region trail to Object-Locked S3 bucket. Immutable record of every AWS API call.
- VPC Flow Logs — full network audit trail → CloudWatch → Logz.io.
- AWS Backup — orchestrates cross-service backup (RDS, S3, EBS).
- AWS Security Hub — NOT used; Logz.io is the primary alert pane.
- Automated cross-tenant isolation tests run in CI on every deploy — prove schema-per-tenant works.
Identity & access (internal staff)
- AWS IAM for human access (not Identity Center / SSO) — IAM users in role-based groups, MFA mandatory, least-privilege policies, no long-lived keys where avoidable (prefer assume-role + temporary creds), root locked down. See the Environments section for detail and the SSO revisit-later note.
- Break-glass procedure for emergency access — audited via CloudTrail
- No direct database access — bastion via AWS Session Manager only, fully audited
- IAM roles for ECS tasks — no long-lived credentials in containers (OIDC, not stored keys)
- Least privilege per service; quarterly access reviews
Email & notifications
- AWS SES — transactional email (alerts, scheduled reports, password resets)
- AWS SNS — internal alerting fanout (email / SMS / Slack via webhook)
- WorkOS sends its own auth-related email (verification, MFA codes) via AuthKit
Infrastructure as Code
- Terraform (HCL) — all AWS infrastructure as code
- Modular: one module per logical layer (vpc, ecs-fargate, rds-postgres, elasticache, s3, logging, observability, iam, secrets)
- Per-environment tfvars scale resources without code changes
- Terraform state in shared-services S3 with S3 native locking (
use_lockfile) — no separate DynamoDB lock table. HashiCorp's S3 backend documentation states that "DynamoDB-based locking is deprecated and will be removed in a future minor version." - GitHub Actions applies via OIDC + cross-account deploy role (no long-lived AWS keys in CI)
Container registry
- Amazon ECR in shared-services account — single registry; every environment pulls the same images
- Image scanning on push (Snyk or ECR-native)
- Lifecycle policy: retain last 50 images, expire older
Disaster recovery
| Failure mode | Target | Mechanism |
|---|---|---|
| Single task / Fargate failure | RTO seconds | ALB removes from rotation; auto-replacement |
| Single AZ failure | RTO < 1 hour | Multi-AZ failover on RDS, ALB, Redis; Fargate cross-AZ spread |
| Data corruption | RPO < 15 min | RDS continuous WAL + 5-min snapshots; PITR |
| Region failure | RTO < 24 hours | Cross-region encrypted backups + Terraform redeploy runbook |
Annual DR drill — tested + documented (SOC 2 mandate). Weekly backup verification — automated restore tests.
Phase 2 — Israel region (AWS Tel Aviv, il-central-1)
Same architecture, deployed as a separate AWS account (set-prod-il) when the first Nimbus-aligned Israeli customer (government / defense / regulated healthcare / banking) signs:
- Same Terraform modules, IL-specific tfvars
- Self-hosted Keycloak instance in IL for Israeli data residency
- Per-tenant routing in NestJS auth middleware (WorkOS for non-Nimbus, Keycloak for Nimbus)
- Logz.io destination TBD at that time (Israel-region option, or self-hosted ELK)
API Strategy
| Layer | Protocol | Why |
|---|---|---|
| Frontend ↔ BFF | REST — current and shipped | One client, and the BFF shapes each response for the view that asked. Type safety end to end comes from the Zod schemas in @set/shared, shared by browser and server, with no codegen step. One action, one endpoint, one audit row. |
| Public/Partner API | REST (@nestjs/swagger / OpenAPI) | Broad compatibility, simple rate limiting, clean audit trail per endpoint |
| Both served from same NestJS server | Shared auth guard + the same tenant guard | Single deployment, consistent security layer |
The frontend protocol is REST. Decided by Ron, 2026-10-08, after being shown what GraphQL would buy and what it would cost. It is not a rejection of GraphQL — it is a decision to make that choice at P3.1, when the dashboards and the compliance graph actually exist, rather than now.
Why REST now, in Ron's own terms:
- There is one client. GraphQL's advantage is letting many different clients each ask for a different shape. The earlier text itself scoped GraphQL to "Internal — Frontend Only", which is exactly the case where that flexibility buys least.
- "One query per view" is already achievable here. Shaping one response for one view is what a BFF is for — the difference is only whether the client composes the shape (GraphQL) or the server does (BFF). With a single client, the server deciding is less code and a smaller attack surface.
- Type safety is already end to end, through the shared Zod schemas, which also validate at runtime at the trust boundary — something generated types do not do.
- The audit rule fits REST. Hard rule 2 is one audit row per action; GraphQL posts every operation to one URL, so "which action was that" becomes work rather than a given.
- GraphQL brings its own mandatory homework — query depth limits, complexity limits and DataLoaders to prevent N+1 — all of which are listed below as requirements and buy nothing until there is a graph to traverse.
P3.1 revisits this, and nothing has to be undone. Adding GraphQL later is additive, never a migration: this page already requires both protocols to be served from the same NestJS server. At that point the two live options are (a) a few view-shaped REST endpoints, which is the BFF pattern continued, or (b) GraphQL alongside REST for graph reads while mutations stay REST and keep the audit trail clean. The cases that would genuinely make GraphQL the right answer, and which should be the trigger to adopt it: more than one client wanting different shapes (a mobile app, partner integrations), a client composing queries nobody anticipated — P4.5's AI platform is the realistic one, since an assistant exploring a compliance graph is a textbook case — or a graph deep enough that writing an endpoint per view becomes the bottleneck.
GraphQL (Internal — Frontend Only) — NOT BUILT. A P3.1 decision, specified here for when it is taken. Nothing below is current, and no design may cite it as a present requirement:
- Powered by Apollo Server via
@nestjs/graphql - Schema-first approach with GraphQL Code Generator → auto-generated TypeScript types + React hooks
- Relay-style cursor pagination (max 100 per page) to prevent DoS — this number governs the GraphQL layer only. It is not a REST page size, and P1.8 was right to stop and ask rather than inherit it: the REST clients list fixed its own page size at 50, because no page in this rule book specified one.
- DataLoaders for batching/deduplication (prevent N+1 queries)
- Query depth/complexity limiting to prevent abuse
- Compliance dashboards, cross-framework mapping, and evidence graphs all benefit from single-query aggregation — the reason to revisit this at P3.1
REST (External — Public API):
- OpenAPI 3.0 spec with Swagger UI documentation
- API key authentication (scoped per tenant, per-endpoint read/write permissions)
- Rate limiting (per-key, per-IP)
- Versioned endpoints (v1, v2)
- Webhook support for real-time event notifications
- Every mutation logged as an audit event
Current implementation
For the Binary Networks platform implementation and verified staging status, see App development and staging. S.E.T is a module inside Binary Networks. This specification remains unchanged.