Blog

Reset a stagelet without its source

How a reset puts a stagelet back to its last successful deploy, with the same images by digest and the same config, on a machine that never gets the source.

A reset puts a stagelet back to its last successful deploy: the same images and the same config, on the same machine. A reviewer clicks Reset on the stagelet’s page in the console. A coding agent calls the MCP tool reset_stagelet. Neither has to know which commit was good.

The stagelet machine never gets the pull request’s source. The agent copies the source only to a build worker. So a reset can’t build anything on the machine, and it does not need to. The excerpts come from the customer agent (stagely-agent) in the public stagely repository, release v0.1.0.

The build worker resolves the project

A deploy starts on a build worker. The agent copies the commit’s source there. Then it runs Compose as the unprivileged user nobody to resolve the project, and checks the result (images.go):

project, err := m.Run(ctx, nil, slices.Concat([]string{"sudo", "-u", "nobody"}, compose, []string{"config", "--format", "json"})...)
if err != nil {
	return Result{}, fmt.Errorf("resolve Compose project: %w", err)
}
if err := CheckProject(project, src); err != nil {
	return Result{}, err
}

docker compose config merges the Compose files, fills in the variables and prints the project in its canonical form. Compose reads the file format, not Stagely. So include, extends and env_file work as Compose defines them. As nobody, Compose can’t read files that only root can read, such as the registry credentials.

CheckProject checks the resolved project against the Compose subset. It refuses bind mounts, privileged containers, host namespaces and other settings that reach past the container. Every variable is filled in at this point, so a variable can’t hide a refused value.

Next, the worker builds each service that has a build section and pushes its image as <registry>/<owner>/<repo>/<service>:<commit>. The agent reads the digest that the registry gave each image. Then it deletes the source from the worker.

The agent compiles a bundle

The function runtime.Compile turns the resolved project into the bundle that the stagelet machine runs:

The bundle holds no secret values. On the machine, Compose reads them from /opt/stagelet/secrets.env, which only root can read.

The agent compiles the bundle from the resolved project that passed the check. Nothing on the stagelet machine reads a Compose file from the repository or resolves the project again.

The machine runs it, then the agent pins every image

In the start step, the agent writes the bundle to /opt/stagelet/compose.json and the secret values to /opt/stagelet/secrets.env. Then it starts the project (runtime.go):

args := compose("up", "--detach", "--remove-orphans", "--wait", "--wait-timeout", "600")
if recreate {
	args = append(args, "--force-recreate")
}

With --wait, Compose returns only when the services are running or healthy. A reset sets recreate to true. A deploy does not.

Built services already run by digest. Other services, such as redis:7-alpine, name their image by tag, and a tag can move to a new image. So after up, the agent asks Docker on the machine which digest it pulled, and writes that digest into the bundle (runtime.go):

if image == "" || strings.Contains(image, "@") {
	continue
}
digest, err := imageDigest(ctx, m, image)
if err != nil {
	return nil, fmt.Errorf("pin the image of service %s: %w", name, err)
}
svc["image"] = digest

Now the bundle names every image by digest. Docker pulls an image by digest as exactly that version. After the migration and the health check, the agent sends the bundle back with the result of the deploy.

Where the bundle lives

On the stagelet machine, the bundle is the file /opt/stagelet/compose.json. Stagely also keeps the bundle of the stagelet’s last successful deploy, so that it can send it back to the agent for a reset.

So Stagely holds what a bundle contains: the plain config values and what your env_file files say. It reads both already when it plans a deploy. Pull request previews in your own cloud lists all that Stagely sees.

What a reset does

When someone asks for a reset, the control plane finds the stagelet’s last successful deploy. It sends the agent a reset job with that deploy’s bundle, public service, port, health path and secret names. The agent then does these steps:

  1. It reads the current value of each secret. A missing secret fails the reset here, before the machine changes.
  2. It checks that the stagelet’s machine still exists.
  3. It signs Docker on the machine in to the registry. Where the cloud allows it, these credentials can only pull.
  4. It writes the bundle and the secret file, and runs docker compose up with --force-recreate.
  5. It requests the health path from inside the machine until it gets a 2xx status, for up to two minutes.

Steps 2 to 4 are in jobs.go:

if !exists {
	return fmt.Errorf("machine %s is gone; deploy the pull request again", name)
}
if err := r.registryLogin(ctx, m, false); err != nil {
	return err
}
result.MachineChanged = true
return runtime.Apply(ctx, m, spec.Bundle, env, true)

Compose recreates a container only when its config or its image changed. A reset can apply the bundle that already runs, for example when the app is in a bad state. Without --force-recreate, Compose would leave every container as it is. --remove-orphans removes the containers of services that a later, failed deploy added.

What a reset does not do

Newest wins

Deploys, resets and deletes of one stagelet can come from a push, a comment, the console, MCP or the TTL. They run one at a time. A newer deploy or reset replaces an older one that is still queued, and the stagelet’s state and commit always follow its newest operation. A deploy or reset that already runs is not stopped. A stagelet that is being deleted refuses new deploys and resets.

A reset takes its bundle when someone asks for it. Take a stagelet whose deploy of commit A succeeded, while a deploy of commit B runs. A reset asked for now carries the bundle of A. It waits until B ends, then applies A. The machine runs A at the end, because the reset was the newest request.

The agent has its own guard. A newer job for the same stagelet waits until the agent’s earlier job there returns, also after a reconnect. So two jobs never work on one machine at once.

What a reset reports

A reset request returns at once. The stagelet’s state becomes deploying, and its commit becomes the commit of the deploy that the reset restores. In get_stagelet, the requested revision shows the kind reset, that commit and the outcome in_progress. When the reset succeeds, the state becomes running, and the applied revision names the reset and the same commit. In the console, the Revision card shows the same.

Two errors come back before any job starts. A stagelet with no successful deploy has nothing to reset to. A stagelet that is being deleted can’t be reset. If a reset fails after it started to apply the bundle, the applied revision shows replaced. The machine may no longer run that revision.

Connect a coding agent lists the MCP tools. Review a pull request explains the console’s cards, and Deploys from pull requests lists every step of a deploy.

More from the blog