Visitar URL original
[BUG] SQS JSON responses over 8 KB use Transfer-Encoding: chunked (real SQS sends Content-Length); aws-sdk v2 on Node 26 hangs on them · Issue #6184 · floci-io/floci · GitHub
Skip to content

[BUG] SQS JSON responses over 8 KB use Transfer-Encoding: chunked (real SQS sends Content-Length); aws-sdk v2 on Node 26 hangs on them #6184

Closed

Description

@bardock

Service

SQS (JSON protocol, X-Amz-Target: AmazonSQS.*)

AWS API Action

ReceiveMessage, and any other SQS JSON response whose body is over 8,191 bytes (a large ListQueues or GetQueueAttributes, for example).

Expected behavior

Content-Length at any size, as real SQS sends. Measured against SQS in us-east-1 over HTTP/1.1 with the JSON protocol: a 25,624-byte ReceiveMessage came back with Content-Length: 25624, and an 802,258-byte one with Content-Length: 802258. Floci's own DynamoDB path already does this, because DynamoDbResponses.withCrc32 serializes the entity to byte[].

Actual behavior

SqsJsonHandler returns Response.ok(ObjectNode). Quarkus REST streams that entity through VertxOutputStream, which switches to Transfer-Encoding: chunked once the default 8,191-byte quarkus.rest.output-buffer-size overflows.

Measured on floci/floci:2.3.0@sha256:e0b292df4dc9559b7b1bf4f51bdda4bb8e269276236438cd03cae661260ea8a9:

One message, response size Framing
8,178 bytes content-length: 8178
8,278 bytes transfer-encoding: chunked

Ten messages of about 2 KB each give a 23,654-byte chunked response. aws-sdk for JavaScript v2 (2.1693.0) then fails to resolve receiveMessage:

Client and server Calls that never resolved (5 s cutoff)
aws-sdk v2, Node 26.10.0, Floci 2.3.0 14/30 (another run: 5/10)
aws-sdk v2, Node 24.21.0, Floci 2.3.0 0/10
@aws-sdk/client-sqs v3, Node 26.10.0, Floci 2.3.0 0/10
aws-sdk v2, Node 26.10.0, LocalStack 4.14.0 (Content-Length: 23404) 0/10
aws-sdk v2, Node 26.10.0, real SQS (Content-Length) 0/30

Why the client hangs: Node 26 made Readable.read() with no size return one buffered chunk at a time (nodejs/node#60441, semver-major). aws-sdk v2 calls read() once per 'readable' event, so when several chunks arrive in one socket read, 'end' never fires. aws-sdk v2 is archived and won't change, and Node treats the new behavior as intended. Others hit the same hang against other servers: balena-io/open-balena-api#2492, serverless/serverless#13971.

Chunked encoding is valid HTTP. But real SQS doesn't chunk these responses, and many Node services still receive through aws-sdk v2. On Node 26 they hang only against Floci, intermittently, and only once a batch crosses 8 KB.

Reproduction

docker run -d -p 4566:4566 floci/floci:2.3.0
aws --endpoint-url http://localhost:4566 --region us-east-1 sqs create-queue --queue-name big
Q=http://localhost:4566/000000000000/big
for i in $(seq 10); do aws --endpoint-url http://localhost:4566 --region us-east-1 sqs send-message --queue-url $Q --message-body "$(head -c 2000 /dev/zero | tr '\0' b)" >/dev/null; done
curl -sv -o /dev/null -X POST http://localhost:4566/ \
  -H 'Content-Type: application/x-amz-json-1.0' -H 'X-Amz-Target: AmazonSQS.ReceiveMessage' \
  -H 'Authorization: AWS4-HMAC-SHA256 Credential=test/20261021/us-east-1/sqs/aws4_request, SignedHeaders=host, Signature=0' \
  --data "{\"QueueUrl\":\"$Q\",\"MaxNumberOfMessages\":10,\"VisibilityTimeout\":0}" 2>&1 | grep -i '^< \(transfer-encoding\|content-length\)'
# < transfer-encoding: chunked

On Node 26 with aws-sdk@2:

const AWS = require('aws-sdk')
const sqs = new AWS.SQS({ endpoint: 'http://localhost:4566', region: 'us-east-1', credentials: { accessKeyId: 'test', secretAccessKey: 'test' } })
;(async () => {
  let hung = 0
  for (let i = 0; i < 30; i++) {
    const r = await Promise.race([
      sqs.receiveMessage({ QueueUrl: 'http://localhost:4566/000000000000/big', MaxNumberOfMessages: 10, VisibilityTimeout: 0 }).promise(),
      new Promise((res) => setTimeout(() => res('HANG'), 5000))
    ])
    if (r === 'HANG') hung++
  }
  console.log(`${hung}/30 hung`) // about half on Node 26, 0 on Node 24
  process.exit(0)
})()

Suggested fix

Serialize SQS JSON responses to byte[] at the AwsJsonController boundary, as DynamoDbResponses already does for DynamoDB. That gives Content-Length at any size. Raising quarkus.rest.output-buffer-size would only move the threshold, and the setting is fixed at build time.

Environment

  • Floci version / image tag: 2.3.0, sha256:e0b292df4dc9559b7b1bf4f51bdda4bb8e269276236438cd03cae661260ea8a9 (SqsJsonHandler has not changed on main since)
  • Java SDK version (if applicable): n/a. Clients: aws-sdk for JavaScript 2.1693.0 on Node 26.10.0, plus Node 24.21.0 and @aws-sdk/client-sqs v3 for comparison
  • How you're running Floci: Docker (Docker Desktop on macOS arm64)

Activity

  1. github-actions commented on Oct 21, 2026

    @github-actions

    👋 Thanks for opening your first issue in Floci!

    A maintainer will triage this and add the right service label. If you included a minimal reproduction and the Floci version, that is everything we need — no action required from you right now.

    Come say hi in Slack if you want to talk it through with maintainers and other contributors. Questions about AWS behaviour, design tradeoffs, and rough proposals are all welcome there.

  2. added
    bugSomething isn't working
    sqsAmazon Simple Queue Service (SQS)
    on Oct 21, 2026
  3. bardock commented on Oct 22, 2026

    @bardock
    Author

    Thanks @JimmyLiuJia for the fix and @pgermosen for the merge!

    I confirmed it with the reproduction from the issue. I built docker/Dockerfile at the merge commit (d7c139c, so the JVM build, not the native image) and ran it next to the 2.3.0 digest as a control. Each run sends 10 messages of about 2 KB, then makes 30 JSON ReceiveMessage calls with MaxNumberOfMessages=10 through aws-sdk 2.1693.0, with a 5 s cutoff:

    Image 23,574-byte response aws-sdk v2, Node 26.10.0 Node 24.21.0
    2.3.0 transfer-encoding: chunked 16/30 hung -
    main @ d7c139c Content-Length: 23574 0/30 hung (2 runs) 0/30 hung

    We'll switch once 2.4.0 (Oct 31 train) is out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsqsAmazon Simple Queue Service (SQS)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions