Serve app assets from S3

[theseanything]

What problem does this pull request solve?

During an ECS rolling deployment, old and new tasks serve traffic together for a few minutes. Each container image only holds the content-hashed assets for its own release, and CloudFront serves /assets/* from the ALB. So on a cache miss a request for a new asset can hit an old task and 404 (which CloudFront then negatively caches for ~10s). Users could see unstyled pages or missing JS during deployment.

The fix is to stop serving assets from the Rails containers altogether. Every deployment now uploads the release's assets to an S3 bucket before the new tasks start, and CloudFront serves /assets/* from that bucket instead of the
apps. Because the bucket keeps the assets from every release, any page a user is looking at, old or new, always references assets that exist, no matter which tasks are currently running. Serving from S3 is behind a setting that is
off by default, so nothing changes for an environment until its bucket has been populated.

This is the same thing Verify did back in 2019: https://technology.blog.gov.uk/2019/04/10/how-gov-uk-verify-maintains-zero-downtime-when-deploying-new-frontend-versions/

This is also helpful as we look at custom branding. If we want to support user uploading assets (e.g. logo), this means this can be done without application deployments.

This is the same pattern used by GOV.UK.

Things to consider when reviewing

• There's deliberately no ALB failover behind the S3 origin: CloudFront must not forward the viewer Host header to an S3 origin (it breaks OAC request signing), and without it a failed-over request wouldn't match any ALB listener rule.
• All three apps share one bucket safely, every file Vite emits is fingerprinted with a content hash. Apps will namespace there assets under /assets/<app-name>/
  • Namespace compiled assets under app name forms-admin#2972
  • Namespace compiled assets under app name forms-runner#2251
  • Namespace compiled assets under app name forms-product-page#1243

Reminders

This PR does change the deployer role (S3 access to the assets buckets, CloudFront origin access control management), so forms/account must be applied before the environment pipelines run.

I have tested this in dev:

• make dev forms/account apply
• make dev forms/environment apply
• make dev forms/pipelines apply

Then deployed all the apps again (populated the assets bucket), then flipped the tfvar to enable serving the assets from S3, no errors.

[Copilot]

Pull request overview

This PR changes the deployment and CloudFront setup so that /assets/* can be served from an S3 “assets bucket” (rather than from the Rails containers behind the ALB), preventing transient 404s and negatively cached misses during ECS rolling deployments. It introduces an opt-in toggle (serve_assets_from_s3) so environments can keep current behaviour until their assets bucket has been populated.

Changes:

• Add a per-environment assets S3 bucket plus CloudFront Origin Access Control (OAC), and optionally route /assets/* to that S3 origin.
• Add a new CodeBuild project and pipeline actions to extract assets from the container image and sync them to S3 during deployments.
• Extend deployer IAM permissions for CloudFront OAC management and access to the assets bucket.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
infra/modules/secure-bucket/outputs.tf	Exposes the S3 bucket regional domain name for use as a CloudFront S3 origin.
infra/modules/environment/variables.tf	Adds serve_assets_from_s3 environment-level toggle.
infra/modules/environment/outputs.tf	Exposes assets_bucket_name from the environment module.
infra/modules/environment/cloudfront.tf	Wires serve_assets_from_s3 into the CloudFront module instantiation.
infra/modules/deployer-access/policy.tf	Grants deployer role CloudFront OAC permissions and assets bucket access.
infra/modules/cloudfront/variables.tf	Adds serve_assets_from_s3 toggle to the CloudFront module.
infra/modules/cloudfront/s3.tf	Creates the secure assets bucket, its CloudFront-read policy, and the OAC resource.
infra/modules/cloudfront/outputs.tf	Outputs assets_bucket_name for consumption by deployment pipelines.
infra/modules/cloudfront/cloudfront.tf	Adds S3 origin and conditionally routes /assets/* to S3 instead of ALB.
infra/deployments/forms/pipelines/sync-assets.tf	Adds a shared CodeBuild project to sync assets from an image into the assets bucket.
infra/deployments/forms/pipelines/deploy-forms-runner-container.tf	Adds a pipeline action to sync assets before ECS deploy.
infra/deployments/forms/pipelines/deploy-forms-product-page-container.tf	Adds a pipeline action to sync assets before ECS deploy.
infra/deployments/forms/pipelines/deploy-forms-admin-container.tf	Adds a pipeline action to sync assets before ECS deploy.
infra/deployments/forms/pipelines/buildspecs/sync-assets/sync-assets.yml	Implements asset extraction from the image and aws s3 sync upload.
infra/deployments/forms/inputs.tf	Adds serve_assets_from_s3 to the environment settings input object.
infra/deployments/forms/environment/outputs.tf	Exposes assets_bucket_name from the environment deployment stack.
infra/deployments/forms/environment/main.tf	Passes serve_assets_from_s3 into the environment module.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

infra/deployments/forms/pipelines/buildspecs/sync-assets/sync-assets.yml:30

• These assets are described as content-hashed/fingerprinted; using Cache-Control: public, max-age=300 will force browsers (and potentially CloudFront, depending on policy) to revalidate frequently, increasing S3/CloudFront load and latency. For immutable, fingerprinted assets it’s typically safe to use a much longer TTL (and optionally immutable).

        aws s3 sync ./assets "s3://${ASSETS_BUCKET}/assets/" \
          --exclude ".vite/*" \
          --cache-control "public, max-age=300" \
          --no-progress