Pull request previews on Azure

Run each pull request's stagelet on its own virtual machine in your Azure subscription. See what stagely setup creates, the service principal and the NSGs.

Stagely gives each GitHub pull request a stagelet: a private, running copy of your app. On Azure, each stagelet runs on its own virtual machine in your subscription. You run stagely setup azure once. After that, the customer agent creates a virtual machine when a pull request deploys, and deletes it when the stagelet ends.

What setup creates

Setup applies a Terraform module with OpenTofu or Terraform. It creates these resources in the location that you choose:

Setup does not manage Azure DNS.

Credentials

Setup needs a service principal, in ARM_TENANT_ID, ARM_CLIENT_ID, ARM_CLIENT_SECRET and ARM_SUBSCRIPTION_ID. The service principal must be able to create resources in the subscription. The Contributor role on the subscription is enough.

By default, setup also gives this service principal to the agent. It is then on the agent host, in /etc/stagely/stagely-agent.env, which only root can read. To give the agent its own service principal instead, set ARM_AGENT_CLIENT_ID and ARM_AGENT_CLIENT_SECRET. A role for the agent shows how to limit that principal to the environment.

No service principal goes to a machine that runs your app.

A role for the agent

The agent needs much less access than setup, and only in the environment’s resource group. This custom role covers the virtual machines, their OS disks and network interfaces, joining the stagelets subnet, reading a golden image, the registry tokens and their passwords, and the long-running operations that these calls start:

{
  "Name": "Stagely agent",
  "Description": "Creates, drives and deletes Stagely stagelet machines and makes short-lived registry tokens in one environment.",
  "Actions": [
    "Microsoft.Resources/subscriptions/resourceGroups/read",
    "Microsoft.Compute/virtualMachines/read",
    "Microsoft.Compute/virtualMachines/write",
    "Microsoft.Compute/virtualMachines/delete",
    "Microsoft.Compute/disks/read",
    "Microsoft.Compute/disks/write",
    "Microsoft.Compute/disks/delete",
    "Microsoft.Compute/images/read",
    "Microsoft.Compute/locations/operations/read",
    "Microsoft.Compute/locations/diskOperations/read",
    "Microsoft.Network/networkInterfaces/read",
    "Microsoft.Network/networkInterfaces/write",
    "Microsoft.Network/networkInterfaces/delete",
    "Microsoft.Network/networkInterfaces/join/action",
    "Microsoft.Network/virtualNetworks/subnets/read",
    "Microsoft.Network/virtualNetworks/subnets/join/action",
    "Microsoft.Network/locations/operations/read",
    "Microsoft.Network/locations/operationResults/read",
    "Microsoft.ContainerRegistry/registries/read",
    "Microsoft.ContainerRegistry/registries/scopeMaps/read",
    "Microsoft.ContainerRegistry/registries/tokens/read",
    "Microsoft.ContainerRegistry/registries/tokens/write",
    "Microsoft.ContainerRegistry/registries/tokens/delete",
    "Microsoft.ContainerRegistry/registries/tokens/operationStatuses/read",
    "Microsoft.ContainerRegistry/registries/generateCredentials/action",
    "Microsoft.ContainerRegistry/locations/operationResults/read"
  ],
  "NotActions": [],
  "AssignableScopes": ["/subscriptions/<subscription id>"]
}

To use it instead of Contributor:

1. Save the role as stagely-agent.json, and create it once:

   az role definition create --role-definition stagely-agent.json

2. Create a service principal for the agent, without a role:

   az ad sp create-for-rbac --name stagely-agent-production

3. Run stagely setup azure with that principal in ARM_AGENT_CLIENT_ID and ARM_AGENT_CLIENT_SECRET, and with the setup principal in the ARM_* variables above. Setup gives the agent only the agent principal.

4. Assign the role on the resource group that setup created:

   az role assignment create \
     --assignee <agent client id> \
     --role "Stagely agent" \
     --scope /subscriptions/<subscription id>/resourceGroups/<environment>-stagely

   If your golden image is in another resource group, assign the role there too.

The agent then cannot change anything outside the environment. If someone takes over the agent host, they get access to that resource group only.

Stagelet machines and build workers

• The agent creates one virtual machine for each stagelet, in the stagelets subnet. Its size is Standard_B2s. Setup has no flag to change this size.
• The agent keeps two build workers, also of size Standard_B2s. They stay when stagelets come and go.
• Each machine starts from Ubuntu 24.04, and cloud-init installs Docker at the first boot. With -golden-image, machines start from a managed image that already has Docker. The stagely repository has the Packer template for this image.
• Machines have no public IP. They reach registries and package mirrors through Azure’s default outbound access, which setup turns on for the stagelets subnet.
• The agent connects to each machine on its private address over SSH, and accepts only the host key that it gave the machine.

Registry

Build workers push images to the Azure container registry. Before each build and each start, the agent creates a new token in the registry, with a password that expires after two hours:

• A build worker’s token can push and pull.
• A stagelet machine’s token can only pull.

The agent removes its expired tokens. The registry’s admin user stays off.

Isolation

• The network security group of the stagelets subnet accepts TCP only from the private addresses of the agent host and the gateway host. It denies the rest of the virtual network. So a stagelet does not accept connections from another stagelet, and it cannot reach the agent host.
• The agent host accepts only SSH, and only from your -operator-cidr ranges.
• The gateway host accepts ports 80 and 443 from anywhere, and SSH from your ranges.
• On each machine, containers cannot open new connections to the instance metadata service at 169.254.169.254. They also cannot reach Azure’s wireserver at 168.63.129.16, except on port 53 for DNS.

Run setup

export ARM_TENANT_ID=<tenant id>
export ARM_CLIENT_ID=<client id>
export ARM_CLIENT_SECRET=<client secret>
export ARM_SUBSCRIPTION_ID=<subscription id>
stagely login
stagely setup azure \
  -environment production \
  -location westeurope \
  -stagelet-domain stagelets.example.com \
  -registry examplestagely \
  -operator-cidr 203.0.113.7/32 \
  -binaries bin/linux-amd64 \
  -acme-email ops@example.com

Setup also takes the flags that every cloud takes. At the end, it prints the gateway’s static public IP. Point DNS at the gateway with an A record.

After setup

1. Add .stagely.yml to your repository.
2. Add config values and secrets.
3. Read the Security model before you add real secrets.