Fail closed: a 50-second limit on access

The stagelet gateway (stagely-gateway) runs in your cloud account, on the only host that is open to the whole internet. It serves the private URL of each stagelet. It holds no cloud credentials and no secret values, and it does not know who is on your team. This post shows how the gateway still keeps each stagelet private, and why it refuses every visitor when it cannot reach the control plane.

Two questions to the control plane

The gateway does not decide who gets in. The control plane decides. The gateway depends on it through an interface with two methods (gateway.go):

type ControlPlane interface {
	ExchangeHandoff(ctx context.Context, token string) (protocol.HandoffResult, error)
	ConfirmSessions(ctx context.Context, ids []string) ([]string, error)
}

The first method trades a pass for a session. The second method asks which sessions are still valid. Both questions go over the one WebSocket connection that the gateway opens to Stagely. The gateway’s record of a session holds a stagelet ID, an expiry time and the time of the last confirmation. It holds no user name, no role and no grant.

A pass that works once, for 30 seconds

A member clicks Open on the stagelet’s page in the console, or a coding agent calls the open_stagelet MCP tool. The control plane checks that the caller is a member of the stagelet’s account. Then it creates a pass for that one stagelet. The pass works for 30 seconds.

The pass goes in the token query parameter of a URL on the stagelet’s own host, at the path /_stagely/handoff. The browser opens that URL. The gateway sends the pass to the control plane and waits at most 10 seconds for the answer. A pass works once, and only within its 30 seconds.

Then the gateway makes one more check (gateway.go):

if !res.OK || res.StageletID != rt.stageletID {
	http.Error(w, "handoff rejected", http.StatusForbidden)
	return
}

The stagelet in the answer must be the stagelet that this host name routes to. The control plane redeems the pass before this check. So a pass that someone opens at the host of another stagelet is spent and gives nothing.

When the pass is good, the gateway stores the session and sets the cookie __Host-stagely_session. The cookie is Secure and HttpOnly, has SameSite=Lax, and expires with the session. A browser accepts a cookie with the __Host- prefix only when it is Secure, has Path=/ and has no Domain attribute. So the browser sends it to that one host name only (RFC 6265bis draft, section 4.1.3.2). Then the gateway redirects the browser to / with 303 See Other, and the pass leaves the address bar. A session lasts at most one hour, and it never outlives the OAuth grant that opened it.

The app never sees the session cookie

The app behind the gateway runs the code of a pull request. Any request to that stagelet’s host that carries the session cookie gets in until the session ends. So the gateway removes the cookie from each request before it proxies the request to the app (gateway.go):

Rewrite: func(pr *httputil.ProxyRequest) {
	pr.SetURL(target)
	pr.SetXForwarded()
	pr.Out.Host = pr.In.Host
	removeCookie(pr.Out, SessionCookie)
},

The app’s own cookies pass through unchanged. HttpOnly keeps the session cookie away from the page’s JavaScript, and removeCookie keeps it away from the app’s server. So code in a pull request cannot read the reviewer’s session, log it or send it to another place.

Confirm every 20 seconds, trust for 50

Two constants set the timing (gateway.go):

const (
	ConfirmWindow = 50 * time.Second
	ConfirmEvery  = 20 * time.Second
)

Every 20 seconds, the gateway sends the IDs of all its sessions in one question. The control plane answers with the IDs that are still valid. The gateway keeps those sessions and records a new confirmation time for each. It deletes every other session and ends the open requests of each deleted session, such as WebSockets and streams.

Each request to a stagelet must pass this check (gateway.go):

if !ok || s.stageletID != rt.stageletID || !now.Before(s.expiresAt) || now.Sub(s.confirmedAt) > g.confirmWindow {
	return nil, nil, false
}

The session must exist, belong to this stagelet, be unexpired, and have a confirmation from the last 50 seconds. Otherwise the visitor gets 401.

Two details keep the window exact. First, each question waits at most half the interval, 10 seconds, so a lost answer never holds up the next question (gateway.go). Second, the confirmation time is the time when the gateway asked, not the time when the answer arrived. A slow answer cannot make the window longer.

What ends access

These events end sessions in the control plane:

• An admin removes a member, or the member leaves the account. The member’s sessions end with the membership.
• Someone revokes an OAuth grant under Connected apps, or a client signs out. The sessions that the grant opened end with it.
• Someone deletes the stagelet, or Stagely deletes it when the pull request closes or merges, or when its TTL ends. Stagely removes access first: it withdraws the route, ends the sessions and waits for the gateway to confirm. Only then does the agent delete the machine.

The control plane does not push these changes to the gateway. The gateway asks again at most 20 seconds after the change, and it waits at most 10 seconds for the answer. That answer leaves the session out, and the session ends at once: new requests get 401 and open requests stop. If no answer arrives, the last confirmation came before the change, and the 50-second window ends the session. Either way, the gateway refuses the session within about 50 seconds of the change. Team members, roles and access shows each of these actions in the console.

When the gateway loses the control plane

The question can fail in three ways. The connection can be down, the control plane can be down, or the control plane can answer that it could not check the sessions. In each case, the gateway confirms no session and revokes none (gateway.go). Its sessions only grow older.

Fifty seconds after the last confirmation, the gateway refuses the session. New requests get 401 with the text open this stagelet through Stagely. At the next round, at most 20 seconds later, the gateway also ends the session’s open requests (gateway.go):

if now.Sub(s.confirmedAt) > g.confirmWindow {
	s.stopRequests()
}

New visitors cannot get in either, because only the control plane can redeem a pass. Without an answer, the handoff fails with 503 and the text Stagely is unreachable, try again. Within about 50 seconds, no request enters a stagelet. Within about 70 seconds, no connection to a stagelet stays open.

The gateway keeps the refused sessions until they expire, and it reconnects to the control plane on its own. The first answer after that confirms each session that is still valid. The reviewer’s next request then works with the same cookie, without a new pass.

The cost of failing closed

This rule has a price. When the gateway cannot reach the control plane, reviewers and coding agents lose access to every stagelet after about 50 seconds. So an outage of the control plane is also an outage of access to stagelets.

The outage does not stop the stagelets. Their machines and containers keep running, and the gateway keeps its route table. Only access stops. When the connection comes back, the sessions that are still valid work again.

In return, the bound holds at all times, also during an outage. A member whom an admin removes while the gateway is cut off from Stagely loses access within about 50 seconds, not when the session expires up to one hour later.

Security model describes the other boundaries between Stagely and your cloud. Review a pull request as a running app shows the reviewer’s side of a pass.