Visitar URL original
Query SSL_pending lazily in ReferenceCountedOpenSslEngine.unwrap by doom369 · Pull Request #18376 · netty/netty · GitHub
Skip to content

Query SSL_pending lazily in ReferenceCountedOpenSslEngine.unwrap - #18376

Open
doom369 wants to merge 2 commits into
netty:4.2from
doom369:lazy-ssl-pending-in-unwrap
Open

doom369 wants to merge 2 commits into
netty:4.2from
doom369:lazy-ssl-pending-in-unwrap

Conversation

@doom369

@doom369 doom369 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Motivation:

ReferenceCountedOpenSslEngine.unwrap(...) calls SSL.sslPending(...) through JNI at the start of every call once the handshake is finished. The value is only read in two places:

  • when the sources are empty (len == 0), to decide between BUFFER_UNDERFLOW and draining plaintext that is still inside the engine;
  • when the loop meets an empty source buffer, for the same decision.

For the common case of one non-empty source the value is never used. In a 300 s production CPU profile of a server handling many small TLS records (Netty 4.2.20, netty-tcnative-boringssl-static 2.0.85) SSL.sslPending accounts for about 1.7% of all samples.

Modification:

  • Do not query SSL_pending up front. Fetch it where it is read: in the len == 0 check, and on the first empty source buffer met in the loop. A SSL_PENDING_UNKNOWN sentinel marks "not queried yet"; SSL_pending never returns a negative value. The refresh after a destination buffer is filled is unchanged.
  • Add OpenSslEngineTest.testUnwrapEmptySourceDrainsPendingPlaintext: it unwraps one full record into a too small destination, then drains the plaintext left inside the engine through an empty source and through an empty source that is one of multiple sources, and checks BUFFER_UNDERFLOW once nothing is left. Neither path had a dedicated test.

Result:

One JNI call less per unwrap in the common case, with the same results.

The value used for an empty source buffer in the loop can now be newer than the one taken at the start of the call. That does not change the outcome: a newer zero skips the buffer directly, where the older non-zero value would have led to a read that returns nothing and then skipped it.

Motivation:

ReferenceCountedOpenSslEngine.unwrap(...) calls SSL.sslPending(...) through JNI
at the start of every call once the handshake is finished. The value is only
read when there is nothing to feed into the BIO (the sources are empty) or
after a destination buffer was filled, so for the common case of one non-empty
source it is never used. On a server handling many small TLS records the call
shows up as ~1.7% of CPU samples in a production profile.

Modification:

- Do not query SSL_pending up front. Fetch it where it is read: when the
  sources are empty, and when an empty source buffer is met in the loop.
- Add a test that unwraps one record into a too small destination and then
  drains the plaintext left inside the engine through an empty source and
  through an empty source that is one of multiple sources.

Result:

One JNI call less per unwrap in the common case.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeEBW1t95tykU91d9HehxQ
wrapFullRecord(...) only accepts MAX_PLAINTEXT_LENGTH bytes, so the test now
wraps a full record and drains it in three parts.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeEBW1t95tykU91d9HehxQ
@testlens-app

testlens-app Bot commented Nov 13, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: e12141f
▶️ Tests: 150712 executed
⚪️ Checks: 26/26 completed


Learn more about TestLens at testlens.app/docs.

@normanmaurer normanmaurer added this to the 4.2.21.Final milestone Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants