feat(SG-3995): add azure private runner stack and autoscaler - #25
Merged
Conversation
ami_cleanup_info.txt was a dated DescribeImages dump from a local destroy run that got committed by accident. Nothing generates it any more. Also ignore tofuplan, which only the per-example gitignores covered.
The standalone-vm rig existed to reproduce a customer host on a stock marketplace image with pinned docker and sg-runner versions. It is not what the modules deploy - no managed identity, no custom image, SSH open by default - and the quickstart covers the supported path. Drops the two pointers to it from the quickstart readme, and the examples/azure gitignore that only existed because the rig was applied from that directory.
Provider requirements are static: there is no conditional required_providers, and count = 0 still installs and configures the provider. So the combined runner_group forced every caller to init and configure both clouds - an AWS user needed azurerm credentials to create an S3 bucket. That is what the skip_credentials_validation flags in its provider block were working around. The cloud resources move into aws/runner_group and azure/runner_group. What is left in runner_group is the platform side - runner group, connector, registration token - which needs no cloud provider at all. It takes a discriminated storage_backend object; both connector variants only ever consumed strings, so nothing cloud-specific crosses the seam. tofu providers now shows no azurerm/azuread under aws/runner_group and no aws under azure/runner_group. The child module also drops its provider block. A module that configures its own providers is a legacy module and cannot take count, for_each or depends_on - the azure quickstart already carries a comment about not being able to depend_on a role assignment for that reason. No moved blocks: nothing is deployed from this tree. Along the way: - azure/runner_group exposes azure_storage_account_id, so the quickstart no longer needs a data lookup to scope its role assignment. - override_names.connector_name now names the azure connector too. It always did in code; only the docs claimed it was AWS-only. - the aws quickstart pinned aws to registry.terraform.io/hashicorp/aws while the modules use hashicorp/aws. Under OpenTofu those resolve to two different providers and both were being locked.
DOCUMENTATION.md is what a user reads while filling the nocode form, so its labels should match the schema titles they actually see. Several had drifted - "Delete EBS Snapshots" for Delete Snapshots, "Automatic AMI Cleanup" for Cleanup AMIs on Destroy, "Additional Security Groups" for Additional Security Group IDs. aws/single_runner was missing Runner Group Token entirely, which is in the schema's required list and is a password field. Outputs tables were also incomplete; adds the 24 outputs that were exposed but undocumented. Navigation: every DOCUMENTATION.md and the packer destroy guide were unreachable - nothing linked to them - and only the examples linked back to the root. Adds a breadcrumb to each module readme and links the stack overviews from the root. The root module tables were also missing aws/single_runner and azure/vmss outright. Adds those, the shared runner_group, and an examples table. Checked: schema fields and variables match 1:1 in all ten modules, every leaf parameter is documented, ui:order is complete, no stray ui_schema keys, and no broken or orphaned links.
aws/packer and azure/packer each carried their own copy of setup.sh and of a build script. The setup copies had drifted about 150 lines apart even though nothing in them is cloud-specific: what actually differed was OS-specific (apt vs yum vs dnf) or the image's admin user, which each copy hardcoded - ec2-user on one side, azureuser on the other. Both now run packer/scripts/setup.sh and packer/scripts/build.sh. The shared setup takes SSH_USERNAME from the caller and branches on OS_FAMILY only, so no cloud branches are left in it. The two build scripts differed only in the -var list they passed to packer build. That list is gone: Terraform sets PKR_VAR_<name> in the environment and Packer reads those natively, so the shared script needs just PACKER_VERSION and PACKER_TEMPLATE and never has to know a cloud's variable names. Along the way, each module gained the knob the other already had: - aws/packer takes ami_name_prefix, matching azure's image_name_prefix. It defaults to the previously hardcoded SG-RUNNER-ami, so existing AMIs still match runner_ami_name_pattern. - azure/packer takes sg_runner.pre_release, which aws already had and the shared setup.sh already knew how to install.
aws/packer takes existing_ami_id and azure/packer takes existing_image_id. Set either one and the module creates nothing at all - no build instance or VM, no Packer download, no manifest parsing, no destroy-time cleanup - and the ami_id / image_id output returns the value it was given. The skip lives in the module rather than in a count on the module call because both modules declare their own provider, which rules out count, for_each and depends_on on the call. A caller wires module.packer.ami_id into its instance once and picks per deployment whether an image gets built. Every build resource is now count-gated on that, including the cleanup hook: the module never deregisters or deletes an image it did not build, so passing one in cannot destroy it on the next destroy. moved blocks cover the resources that gained a count, so a state written before this does not re-key and rebuild.
The runner group and the connector were named
{prefix}-runner-group-{account_id} and
{prefix}-private-runner-backend-{account_id}. On Azure that spent 36 of
the name's characters on a subscription ID, which told a reader nothing
they could not get from the deployment itself.
Both are now {global_prefix}-{name}, or just {name} when global_prefix
is empty. The name half is override_names.runner_group_name; left empty
it is a 6-character random string, which is all the uniqueness a runner
group needs. The connector shares the runner group's name - they live in
separate API namespaces (/integrations/ vs runnergroups/), so there is
nothing to clash with, and the pair is always created 1:1.
What used to be in the name is a tag instead. The platform models tags
as a flat list of strings with no keys, capped at 10, so the shared
runner_group module always sets three - "StackGuardian Private Runner",
"Managed by IaC", and the cloud - and takes up to seven more from the
caller. The per-cloud modules pass the account or subscription ID, the
region and the naming prefix.
Dropped from the tag list: the org name (a runner group only ever lives
in one org) and the runner group's own name. Both were pure
duplication. "Managed by IaC" is deliberately tool-agnostic - this runs
under both OpenTofu and Terraform, and the tag's job is "do not
hand-edit this in the console".
override_names.include_org_in_prefix goes with it. It only ever fed the
prefix, and the org is not in the name any more. The VM-side modules
keep their own copy of the field.
Azure resource names are untouched: the resource group, storage account
and Entra ID application still derive from the sanitized prefix.
It never worked. The VM name, the OS disk name and the NIC are all singular, and a NIC can only ever be attached to one VM, so Azure rejects the replacement with "a resource with the ID ... already exists" before the original is torn down - the flag turned a destroy-then-create into a failed apply. Replacements are destroy-then-create now, which means the runner is briefly offline while it is rebuilt. The comment says so, so the next reader does not put the flag back.
Both quickstarts used to build their own networking - the azure one created a VNet and subnet, the aws one took a bare vpc_id and public_subnet_id and assumed a public path out. Neither matches how these get deployed: the network already exists, and it is not the example's to manage. Each now takes a network block naming what to attach to, looks it up with data sources, and creates only what is bound to the instance itself - the security group or NSG, the NIC, and a public IP unless network.associate_public_ip is turned off. A destroy leaves the network exactly as it was. Two plan-time checks, because Azure and AWS both report the mistake late and badly: - the aws subnet has a postcondition that it belongs to the given VPC, which otherwise only surfaces when the instance is created. - the azure VNet has one that its region matches azure_location. A NIC can only join a subnet in its own region and a VM can only boot from an image in its own region, and Azure reports the mismatch as "InvalidResourceReference ... was not found" on the NIC, halfway through the apply and after the image build. Both also gained a way to skip the build: ami_id / vm_image_id feed the packer module's existing_ami_id / existing_image_id, so one image can back several deployments, or be pinned to a known-good build. The ami_id / image_id output reports what the runner actually booted from either way. Along the way: - ssh_command falls back to the private IP when no public IP is attached, instead of rendering "ssh user@" with nothing after it. - the azure quickstart passes sg_runner through to the image build. - override_names is destructured for the runner group module, which no longer takes include_org_in_prefix. The VM modules still do. - subnet_id is exposed as an output, so what was attached to is visible without reading the tfvars.
The quickstarts build an image and a runner in one apply, which is the wrong shape when the image is the deliverable - one team bakes it, and several deployments boot from it. examples/aws/packer and examples/azure/packer run just the packer module. No runner group, no connector, no instance. They output the ami_id / image_id to feed into the quickstarts' new ami_id / vm_image_id, plus the cleanup commands for removing the image by hand. Both take an existing network the same way the quickstarts now do. The aws one builds in a subnet you name; the azure one defaults to Packer's own throwaway networking and takes a VNet only if the build has to sit inside yours. Neither declares a provider - the packer modules configure their own - so these are thin root modules: a module call, variables, and outputs.
existing_ami_id and existing_image_id were reachable only from HCL. A nocode deployment of either packer template had no way to skip the build, even though the variable, the validation and the readme section were all there. Both are now the first field in the form, since they decide whether anything below them applies at all. The patterns allow an empty value, matching the variables: empty means build.
DOCUMENTATION.md is what a user reads while filling the nocode form, so
it has to track the schema. Three changes moved out from under it.
Runner group naming. Both runner_group docs still listed *Include
Organization Name in Prefix*, a field that no longer exists, and
described the other two as "Overrides" defaulting to "Auto-generated".
They are now the name half of {prefix}-{name}, defaulting to a 6-char
random string and to the runner group's own name. The Resource Naming
notes still claimed SG_RUNNER-{type}-{account_id} / {subscription_id}.
Tags. Both overviews claimed the default tags were "StackGuardian
Private Runner", the runner group name, and the organization name. The
last two were dropped deliberately; what is applied now is the purpose
marker, "Managed by IaC", the cloud, the account or subscription ID,
the prefix and the region. Each doc gained a Tags note saying so, and
saying why the org and the group's own name are absent.
New packer inputs. existing_ami_id, existing_image_id, ami_name_prefix
and sg_runner.pre_release were all undocumented. Each is in its
module's table and in the matching stack table, and both packer docs
gained a Skipping the Build note - including the warning about setting
it on a deployment that already built an image, where the destroy-time
cleanup would delete the very image being passed in.
The include_org_in_prefix rows under Template 3 and Template 4 stay:
the autoscaling group, VMSS and autoscaler modules still take it.
Dropping include_org_in_prefix left it as effective_prefix = var.override_names.global_prefix - an alias that computed nothing and, worse, kept promising it did. "Effective" meant "the prefix after the org name is folded in", and nothing folds in any more. Both modules now read var.override_names.global_prefix directly, which is longer at the call site but says where the value comes from without a trip through locals.tf. Purely mechanical: the local was a one-line alias, so every name produced is byte-for-byte what it was. The other six modules keep their effective_prefix - autoscaling_group, autoscaler, single_runner, vmss, azure_runner and the azure autoscaler still take include_org_in_prefix, so theirs is a real computation. Also fixes the comment above azure's sanitized_prefix, which claimed the org name flowed into Azure resource names.
hllvc
marked this pull request as ready for review
August 28, 2026 10:15
joscheuerer
approved these changes
Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a complete Azure private runner stack — managed image build, runner group with an Entra ID OIDC connector, VM Scale Set, single-VM runner, and a Function App autoscaler — bringing Azure to parity with the existing AWS stack. Along the way the shared
runner_groupmodule is split per cloud, the Packer scripts are deduplicated across both clouds, and both quickstart examples move to a bring-your-own-network model.Motivation & Context
Private runners were AWS-only. Customers running on Azure had no supported path, and the pieces that did exist could not simply be duplicated:
required_providers, andcount = 0still installs and configures a provider. A combinedrunner_grouptherefore forced every AWS user to init and configureazurerm/azureadjust to create an S3 bucket — which is what theskip_credentials_validationflags in its provider block were working around.aws/packerandazure/packereach carried their ownsetup.sh, and the two copies had drifted roughly 150 lines apart even though nothing in them was genuinely cloud-specific.No GitHub issues are referenced by the commits on this branch; tracking is under SG-3995.
Changes Made
New Azure stack (
azure/)azure/packer— builds a managed image with Docker, jq, cron, sg-runner and optional Terraform/OpenTofu. The image ID is recorded in state, so it builds on the first apply and is reused after that;packer_config.rebuild_image_tokenforces a rebuild.azure/runner_group— runner group and connector on the platform, plus a resource group and storage account for workflow artifacts. Authenticates via an Entra ID OIDC connector, so no long-lived secret is stored on the platform. TheStorage Blob Data Readerassignment is optional for identities that only holdContributor.azure/vmss— Linux VM Scale Set whose instances self-register as runners on boot.azure/azure_runner— single-VM alternative for a fixed-size footprint or a pilot.azure/autoscaler— Azure Function (Flex Consumption, Python 3.11) on a one-minute timer that watches StackGuardian job queues and adjusts VMSS capacity. Ships Application Insights, blob-backed cooldown state, and the managed-identity role assignments it needs.azure/DOCUMENTATION.md— stack overview tying the five Azure templates together.Shared
runner_groupsplit per cloudrunner_groupis reduced to the platform side — runner group, connector, registration token — which needs no cloud provider at all. It now takes a discriminatedstorage_backendobject; both connector variants only ever consumed strings, so nothing cloud-specific crosses the seam.aws/runner_group(new) andazure/runner_group.tofu providersnow shows noazurerm/azureadunderaws/runner_group, and noawsunderazure/runner_group.count,for_eachordepends_on.Packer: shared scripts and image reuse
packer/scripts/build.shandpacker/scripts/setup.shreplace the four per-cloud copies. The shared setup takesSSH_USERNAMEfrom the caller and branches onOS_FAMILYonly. Terraform setsPKR_VAR_<name>in the environment and Packer reads those natively, so the build script no longer needs to know either cloud's variable list.existing_ami_id/existing_image_idlet either module hand back an image you already have: no build instance or VM, no Packer download, no destroy-time cleanup. The skip lives in the module rather than acounton the module call, because both modules declare their own provider.movedblocks cover the resources that gained acount.ami_name_prefixon AWS,sg_runner.pre_releaseon Azure.Runner group naming and tags
{global_prefix}-{name}, or just{name}when the prefix is empty. Left unset, the name half is a 6-character random string. Previously they were{prefix}-runner-group-{account_id}— on Azure that spent 36 characters on a subscription ID.StackGuardian Private Runner,Managed by IaC, the cloud) and accepts up to seven from the caller.override_names.include_org_in_prefixis removed from the runner group modules — it only ever fed the prefix, and the org is not in the name any more. The VM-side modules still take it.Examples
examples/aws/quickstartandexamples/azure/quickstart— full working runner in one apply. Both now attach to an existing VPC/VNet and subnet instead of creating networking, and both accept a prebuiltami_id/vm_image_idto skip the build. Plan-time postconditions catch a subnet in the wrong VPC (AWS) and a VNet in the wrong region (Azure), which Azure otherwise reports as a misleadingInvalidResourceReferenceon the NIC after the image build.examples/aws/packerandexamples/azure/packer— image-only roots, for baking once and feeding the resulting ID into other deployments.AWS side and housekeeping
aws/packer,aws/runner_group,aws/single_runner,aws/autoscaling_groupandaws/autoscalerall move to the shared patterns.azure/azure_runnerdropscreate_before_destroy: the VM name, OS disk name and NIC are all singular and a NIC can only attach to one VM, so Azure rejected the replacement before the original was torn down.input_schema.json/ui_schema.json/README.md/DOCUMENTATION.mdrealigned across every changed template, including the new Packer inputs and the new naming and tag behaviour..gitignorepicks uptofuplanandami_cleanup_info.txt; a committed AMI dump was removed.Testing
tofu validatepasses on all four example roots, which exercisesaws/packer,azure/packer,aws/runner_group,azure/runner_group, the sharedrunner_group,aws/single_runnerandazure/azure_runnertofu fmt -check -recursiveclean across the treeui:ordertofu providersconfirms no cross-cloud provider leaks after the runner group splitrebuild_ami_token/rebuild_image_tokenand confirm exactly one rebuild, with the superseded image cleaned upexisting_ami_id/existing_image_idon a fresh deployment and confirm nothing is builtRisks & Edge Cases
{prefix}-runner-group-{account_id}become{prefix}-{random}, replacing both platform records. Nomoved/import path is provided — the branch's position is that nothing is deployed from this tree yet. Worth confirming before merge if that is no longer true.include_org_in_prefixis gone from the runner group inputs. Any tfvars or saved nocode form still setting it on those two templates will fail validation.existing_ami_id/existing_image_idare meant for a fresh state. Adding either to a deployment that already built an image tears down the build records, and the destroy-time cleanup deletes the image that was built — including when that is the image being passed in. Both READMEs document thetofu state rmworkaround.create_before_destroynever actually worked here.RoleAssignmentExists. Re-import rather than apply.terraform destroy.vpc_id/public_subnet_idare replaced by a requirednetworkblock, andcreate_network/ NAT gateway /proxy_urlare only reachable through the underlying modules.Deployment Notes
Storage Blob Data Readerassignment and the autoscaler's VMSS/storage/network assignments requiresMicrosoft.Authorization/roleAssignments/write(Owner or User Access Administrator). With onlyContributor, setcreate_role_assignments = false/create_blob_reader_role_assignment = falseand create them out of band — runners cannot read state until they exist.azure/autoscaler/scripts/deploy_function.shas part of the apply; the runner needs outbound access to fetch the function source, and the source location is configurable.azurermis pinned to 4.x across theazure/*modules.packer/scripts/build.shfrom the repo root, two levels above each packer module. Any deployment that vendors a single module directory rather than the whole tree needs to pick up that shared directory too.