HttpApiSchema.Upgrade: allow a request payload and headers on the 101 response #9682

Activity

1. fubhy

   added

   enhancementNew feature or requestNew feature or request

   docker-made-me-do-itBlame the Docker Engine APIBlame the Docker Engine API

   on Oct 21, 2026
2. effect-bot

   self-assigned this

   on Oct 21, 2026
3. tim-smart commented on Oct 21, 2026

   

   tim-smart

   on Oct 21, 2026

   Contributor

   More actions

   Thanks for the detailed report. We would like to take this as a follow-up to #9339. Below is the design we intend to implement, plus three questions for you at the end, since the handler shape should be settled before Upgrade ships in a release.

   One finding changes the Node side: Node 25 and older do not parse the body of an upgrade request. Node 26 does (nodejs/node#60016, semver-major). We reproduced this with raw TCP probes against Node 22.23, 24.21 and 26.8:

   Node	head at the upgrade event	request body on req
   26	bytes after the body	parsed, including chunked
   22 / 24	body bytes (chunk framing included) followed by any raw bytes	empty

   Your report mentions Node 26, Deno 2.8 and Bun 1.3, which matches. @effect/platform-node declares node >=18, so the adapter has to handle both behaviours. This also means today's no-payload Upgrade endpoint already leaks a stray request body into the socket on Node ≤25.

   Proposed design

   1. Headers are declared on Upgrade, not through WithHeaders. Your proposal used WithHeaders(Upgrade, H). We prefer Upgrade({ protocol, headers }): every consumer (handler type, builder, client, OpenAPI) already special-cases Upgrade, and the WithHeaders value type would invite returning withHeaders(...) from the handler, which arrives after the 101 is already committed. The headers still use the same string-tree codec as WithHeaders, so OpenAPI and the client reuse it.

   const start = HttpApiEndpoint.post("start", "/exec/:id/start", {
     params: { id: Schema.String },
     payload: Schema.Struct({ Detach: Schema.Boolean, Tty: Schema.Boolean }),
     success: HttpApiSchema.Upgrade({
       protocol: "tcp",
       headers: {
         "content-type": Schema.Literals([
           "application/vnd.docker.raw-stream",
           "application/vnd.docker.multiplexed-stream"
         ])
       }
     })
   })
   
   // server: `socket` is replaced by a deferred `upgrade` function
   handlers.handle("start", ({ params, payload, upgrade }) =>
     Effect.gen(function*() {
       const exec = yield* lookup(params.id) // declared errors still work, no 101 is sent
       const socket = yield* upgrade({
         headers: {
           "content-type": payload.Tty
             ? "application/vnd.docker.raw-stream"
             : "application/vnd.docker.multiplexed-stream"
         }
       })
       // ...
     }))
   
   // client
   const { headers, socket } = yield* client.exec.start({
     params: { id },
     payload: { Detach: false, Tty: true }
   })
   headers["content-type"] // typed literal union; plain `Headers` when none are declared

   2. Handler shape. upgrade is always a function: upgrade() when no headers are declared, upgrade({ headers }) when they are, with the argument typed from Upgrade<H>. The 101 is still written when the socket reader is acquired, so headers can depend on the decoded payload (your Tty case). A header encoding failure takes the same 500 path as any bad response header today. The handler keeps returning void.

   3. Payload. Buffered payloads (JSON, text, urlencoded, buffered multipart) become allowed. StreamUint8Array, StreamNdjson and MultipartStream stay rejected with a clear error: the 101 can only follow the end of the request message, so a streaming payload would have to be fully drained before the socket exists anyway.

   4. Request pipeline order. Params, headers, query, then the 426 Upgrade Required check, then payload decoding, then the handler. An invalid body gets the normal 400 with no 101.

   5. HttpServerRequest.upgradeWith({ headers }) is added next to the existing upgrade getter, which becomes sugar for upgradeWith({}). WebSocket upgrades with non-empty headers fail with an HttpServerError rather than silently dropping them. WebSocket handshake headers stay out of scope.

   6. Node adapter.

   • Custom headers are validated when upgradeWith runs, using http.validateHeaderName/Value. connection, upgrade, content-length and transfer-encoding are rejected, not stripped. Custom headers are written after the fixed two.
   • Node 26: upgrade can fire while the body is still arriving. The 101 write and reader attach wait for the request body to end; an unconsumed body is drained within the usual size limit, otherwise the upgrade fails.
   • Node ≤25: the adapter frames the body itself from head plus the socket, Content-Length only, within the payload limit, unshifting leftovers. Chunked upgrade bodies get 411 Length Required. Docker clients and our clients send Content-Length for buffered bodies.
   • The 101 response that middleware and logging observe carries the sent headers.

   7. Client and OpenAPI. The payload is encoded and sent as for any endpoint. The client returns { headers, socket } with headers decoded through the declared schema. OpenAPI documents the request body as usual and lists connection, upgrade and the declared headers on the 101.

   Questions for you

   1. Are you fine with Upgrade({ protocol, headers }) instead of WithHeaders(Upgrade, H)? Both give you the typed { headers, socket } on the client.
   2. Do you need upgrade bodies to work on Node 25 and older? Our plan is to support them with Content-Length framing and reject chunked bodies with 411 there. The alternative is requiring Node 26 for upgrade bodies and failing explicitly on older runtimes.
   3. Is a bare upgrade() call acceptable for endpoints without declared headers, or would you rather keep the pre-acquired socket in that case? We prefer the uniform function so there is one shape to document.

   Anything else from your Docker use case that this design would block, let us know here.

   👀React with 👀1Reacted by Sebastian Lorenz
4. fubhy commented on Oct 21, 2026

   

   fubhy

   on Oct 21, 2026

   MemberAuthor

   More actions

   1. Okay
   2. Node v25 is already EOL, I'd argue we should require Node v26 at this point
   3. Yes
5. tim-smart commented on Oct 21, 2026

   

   tim-smart

   on Oct 21, 2026

   Contributor

   More actions

   Node 22 and 24 are still under LTS

   
6. fubhy commented on Oct 21, 2026

   

   fubhy

   on Oct 21, 2026

   MemberAuthor

   More actions

   2. Tim is boring. But yes, your proposed fallback is acceptable.
7. tim-smart

   closed this as completedin #9751on Oct 23, 2026