Supply Chain & CI/CD Security
Development is performed by a third-party firm: their developers are GitHub collaborators with no AWS access. They can only propose changes; nothing reaches production without S.E.T-team approval. This section defines how a code change travels safely from a developer's laptop to production, and the controls at each stage.
Threat model
Defending against the 2025–2026 supply-chain attack wave: Shai-Hulud (self-replicating npm worm via postinstall, weaponized TruffleHog to steal creds), tj-actions/changed-files + reviewdog (poisoned Action tags dumping runner secrets), TanStack (OIDC token extraction + cache poisoning), TeamPCP (Trivy/Checkmarx Actions compromise that ran curl 169.254.169.254 for IMDS creds), and assorted hijacked packages (Axios, node-ipc, Bitwarden CLI). Also: a compromised contractor laptop, a rogue/compromised contractor, and the contractor firm itself being breached.
The two zones
- Untrusted zone — the contractor's laptop. We cannot control it; we assume it could be compromised and rely on nothing here.
- Our turf — everything from GitHub onward. Runs on systems we control; gates cannot be bypassed.
The border is the push to GitHub. Real enforcement begins there.
Pipeline at a glance
A plain-language, one-page view of how a change travels from a laptop to production and what protects it at each step. The stage-by-stage detail follows in the table below.
The words, simply:
| Word | What it means |
|---|---|
| commit | Save a labeled snapshot of your changes (a checkpoint). |
| push | Upload your commits from the laptop up to GitHub. |
| pull | Download the latest commits down from GitHub to your laptop. |
| branch | A separate lane to work in without disturbing main (the live version). |
| pull request (PR) | A "please review and merge my branch" request — where checks run and a human reviews. |
| merge | Combine an approved branch into main. |
| runner | A fresh, throwaway cloud computer GitHub spins up to run the automated checks, then discards. |
The five ideas that protect the whole thing:
- Keep untrusted code away from anything valuable — the checking runner holds no keys.
- Never store permanent keys — deploys borrow a short-lived OIDC token that expires in ~1 hour.
- Least privilege — every token/role can do the minimum and nothing more.
- No bypass — every change is a reviewed pull request; even the owners go through it.
- Pin everything, scan everything, in layers — SHA-pinned Actions and many scanners, so one miss is never fatal.
One sentence: a developer saves changes (commit), uploads them (push) into a lane (branch), and asks to add them (pull request); a throwaway cloud computer (runner) with no keys scans everything; a human approves; then a separate runner borrows a one-hour, tightly-scoped key to ship a signed image — and nobody can skip the gates.
Third-party developer access model
- Contractors are Write-only GitHub collaborators (never Admin/Maintain); only the S.E.T team can change settings, branch protection, or add collaborators
- Mandatory MFA enforced at org level; no shared accounts (each developer uses their own named identity)
- CODEOWNERS names the S.E.T team as required approver for sensitive files:
package-lock.json,.github/workflows/**,infra/**(Terraform), production config, schema migrations - No AWS access — contractors never hold cloud credentials; deploys run via OIDC under S.E.T's control
- Monthly GitHub audit-log review; quarterly collaborator-list reconciliation against the firm's current roster
- Contract requirements: DPA (Amendment 13), NDA, security clauses (MFA, no shared accounts, no committed secrets, breach notification), 24–48h offboarding SLA, right-to-audit, IP assignment (work-for-hire), cyber-insurance
Stage-by-stage controls
| Stage | What happens | Controls | Enforced? |
|---|---|---|---|
| 0. Foundation | S.E.T team builds the secure skeleton ("the cage") before contractors are added | Repo + branch protection + CODEOWNERS + pre-built CI workflows + AWS OIDC + org policies | — |
| 1. Laptop | Contractor writes code, pulls npm packages | Untrusted — rely on nothing. Contract requires laptop hygiene + 2FA. Optional dev-container sandbox | No (untrusted zone) |
| 2. Commit | Local snapshot | lefthook → Gitleaks + ESLint (fast feedback only; bypassable) | No (convenience) |
| 3. Push | Upload to a draft branch | GitHub Push Protection (secrets); branch protection blocks direct push to main/develop | Yes |
| 4. Pull Request | Validation job runs — holds zero credentials | Harden-Runner egress block · npm ci --ignore-scripts · TruffleHog (--only-verified) · OSV-Scanner · Trivy · Socket Firewall · Semgrep · zizmor + actionlint · tests + cross-tenant isolation tests · container build + Trivy image scan | Yes |
| 5. Human review | S.E.T team reviews + approves | Required reviewer (contractor can't self-approve); CODEOWNERS gates sensitive files | Yes |
| 6. Merge → nonprod | Deploy job (separate from validation) gets short-lived OIDC creds, deploys to nonprod | Job separation · short-lived OIDC · least-privilege deploy role · SHA-pinned steps only | Yes |
| 7. Test nonprod | QA + demo | Separate AWS account; no real customer data | Yes |
| 8. Promote → main | Deliberate second approval | GitHub Environment "production" required-reviewer gate (recommend two — four-eyes) | Yes |
| 9. Deploy prod | Same tested image deploys | Rolling deploy + circuit breaker + auto-rollback; S.E.T creds only | Yes |
| 10. Runtime | Live + monitored | VPC Flow Logs + GuardDuty → Logz.io; alerts → PagerDuty | Yes |
Core CI/CD security principles (locked)
- OIDC short-lived credentials only — no long-lived AWS keys stored anywhere. A leaked token dies in ~1 hour.
- Job separation — untrusted contractor code (install/build/test) never runs in the same job that holds AWS credentials. The validation job has zero secrets; the deploy job runs only trusted, SHA-pinned, post-review steps.
- Least-privilege deploy role — even a stolen/used credential cannot create IAM backdoors or copy data cross-account; the role can only perform narrow deploy actions.
- GitHub-hosted runners only — they run on Azure, so
curl 169.254.169.254returns no AWS credentials. No self-hosted runners on AWS without IMDSv2 + hop-limit-1 + network-namespace IMDS blocking + ephemeral runners. - All third-party Actions pinned by 40-char commit SHA (via
pinact); a CI check blocks unpinned actions. Org policy: Allowed Actions = selected + verified creators only. - Layered exfiltration defense — egress allowlist (Harden-Runner) + DNS controls + least-privilege + short-lived creds + GuardDuty detection. No single control is relied on (egress filtering has known bypasses — see caveats).
Locked tool stack (all free; validated for stars / maintenance / reliability 2026-05-24)
| Category | Tool | License | Role |
|---|---|---|---|
| Git hook manager | lefthook | MIT | Runs local pre-commit checks (replaces Husky — Husky stale since Nov 2024) |
| Secret scan (pre-commit) | Gitleaks | MIT | Fast local secret scan |
| Lint | ESLint + security plugins | MIT | Code quality + dangerous-pattern lint |
| Secret scan (CI) | TruffleHog --only-verified | AGPL-3.0 | Verified secret scan on PR (blocks merge) |
| Dependency vuln scan | OSV-Scanner | Apache-2.0 | npm vuln + malware advisories (caught Shai-Hulud cleanly) |
| Backup vuln + image scan | Trivy | Apache-2.0 | Container image CVE scan + secondary dep scan |
| SAST | Semgrep OSS | LGPL-2.1 | Dangerous-code analysis on every PR |
| Workflow analysis | zizmor + actionlint | MIT / Apache-2.0 | zizmor = security; actionlint = syntax/shell (run both) |
| Runner hardening | Harden-Runner ≥ v2.19.4 | Apache-2.0 | Egress monitoring/block (detection + defense-in-depth) |
| SBOM | Syft + cyclonedx-npm | Apache-2.0 | Build SBOMs (container + npm) for SOC 2 evidence |
| Pin Actions by SHA | pinact | MIT | Auto-pins Actions (ratchet rejected — stale since Jun 2025) |
| Posture scoring | OpenSSF Scorecard | Apache-2.0 | Weekly scheduled posture score (non-blocking) |
| Artifact signing | cosign (Sigstore) | Apache-2.0 | Keyless OIDC signing of images |
| Install-time block | Socket Firewall Free | Free tier | Blocks confirmed-malicious packages at install (early-warning) |
| Lockfile integrity | npm ci --ignore-scripts | built-in | Lockfile match + no postinstall execution |
| Auto dependency updates | Dependabot | GitHub | Patch/minor auto-PRs; major requires manual review |
| Secret push block | GitHub Push Protection | GitHub | Blocks secrets at push (Stage 3) |
| DAST (passive + active) | OWASP ZAP | Apache-2.0 | Per-PR baseline + nightly full scan against nonprod; findings flow into Vulnerability Management per the DAST & Penetration Testing Policy |
| DAST (templated CVE / vuln) | Nuclei | MIT | Weekly templated vulnerability coverage against nonprod; complements ZAP |
Deliberately avoided paid traps: CodeQL (needs paid GitHub Advanced Security on private repos) → use Semgrep instead; GitHub native secret scanning (paid for private) → covered by TruffleHog + Gitleaks. Grype rejected (false positives on npm devDependencies in Shai-Hulud post-mortems). Docker Scout rejected (free tier caps at 1 repo).
Reliability caveats (known, accepted)
- Harden-Runner free-tier egress has 2026 bypass CVEs (CVE-2026-32946/32947 — DNS-over-TCP / DNS-over-HTTPS). Fixed in v2.16.0+; pin ≥ v2.19.4. Treat free-tier egress filtering as detection + defense-in-depth, NOT a hard boundary — which is why it's layered with job separation, least-privilege, and short-lived creds.
- Socket Firewall Free has a
.swf.configbypass and only blocks confirmed malware (unconfirmed = warn-only). Strong early-warning layer, not a guarantee; paired withnpm ci --ignore-scripts. - Semgrep OSS has a large open-issue backlog (mostly rule requests, not bugs) — still the best free SAST.
SOC 2 / Amendment 13 evidence retention
- SBOMs (every build) — retained ≥ 12 months
- Scan reports (TruffleHog, OSV-Scanner, Trivy, Semgrep) — retained as PR check evidence
- Signed artifacts (cosign) — provenance trail
- OpenSSF Scorecard history — posture over time
- GitHub deployment + approval logs — automatic change-management evidence (who approved which production deploy)
- All retained in an immutable store for auditor access — deployed as the WORM S3 evidence archive; see Evidence Collection & Retention §8.
As-built implementation (2026-08) — the reusable security gate & org rollout
Status: In place. The sections above describe the design. This section documents what is actually deployed across the GitHub org today, so an auditor can reconcile policy to running control.
One reusable gate, called by every repo
The scanner suite is not copy-pasted per repo. It lives once as a reusable workflow —
set-platform/.github/workflows/security-core.yml (on: workflow_call) — and every repo
opts in with a tiny caller (.github/workflows/security.yml) that pins it by 40-char
commit SHA and passes secrets: inherit. Update the gate in one place; every repo inherits it.
security-core.yml runs two jobs, each ending in a per-check Slack alert to #ci-cd
that names exactly what failed, then fails the job so the merge stays blocked:
| Job | Checks (all version-pinned) |
|---|---|
| Security scan | TruffleHog v3.97.1 (--only-verified) · OSV-Scanner v2.5.1 · Trivy v0.74.0 (fs: vuln + secret) · Semgrep 1.175.0 |
| Workflow hardening | actionlint v1.7.12 · zizmor 1.29.0 · pin-ratchet (all Actions SHA-pinned, images pinned) |
Separately, each repo also runs log-archive.yml, which ships the completed run's logs +
artifacts to the WORM evidence bucket — see Evidence Collection & Retention §8.
Three layers of enforcement (what's org-wide vs per-repo)
- Org-wide (automatic): 2FA required, default access
none, members can't create private repos. - Not available: org-level rulesets that would enforce branch protection across all repos are an Enterprise feature; on the current Team plan they cannot be used, so branch protection is configured per repo.
- Branch protection is per-repo and solo-appropriate: PRs are required and direct pushes to
mainare blocked, but because the team is small, CI (the gate above) is the reviewer — required human approvals are set to 0 where a second reviewer isn't realistic, rather than leaving an unmet 2-approver rule that would deadlock merges.set-platformadditionally keeps admin-enforcement on with the workflow-hardening check required.
As-built deviations from the original design (honest reconciliation)
| Design (above) | As-built | Why |
|---|---|---|
| Dependabot for auto-updates | Removed | Redundant with OSV-Scanner + Trivy dependency scanning; fewer moving parts |
| Runtime alerts → Logz.io / PagerDuty | Grafana + Loki (infra) and Slack #ci-cd (CI, per-check) | Matches the actually-deployed observability stack |
| Two-reviewer production gate | Compensated by the CI gate + the three-review pre-push discipline (security / over-engineering / dead-code) | Two-person company — a mandatory second human approver isn't always feasible; machine review is the standing gate |
| CI logs to a generic "immutable store" | WORM S3 iontwrks-ci-evidence (eu-north-1, acct 429134227608) via OIDC | Concretely deployed — see Evidence §8 |