Visitar URL original
PumpReader.createInputStream(...) returns EOF for supplementary code points · Issue #658 · jline/jline3 · GitHub
Skip to content

PumpReader.createInputStream(...) returns EOF for supplementary code points #658

Closed

Description

@Marcono1234

The InputStream created using org.jline.utils.PumpReader.createInputStream(Charset) returns EOF (-1) when encountering a supplementary code point (i.e. > U+FFFF) in the input, e.g.:

PumpReader reader = new PumpReader();
reader.getWriter().append("\uD83D\uDE0Atest");
InputStream in = reader.createInputStream(StandardCharsets.UTF_8);
in.read();

The reason for this is that the buffer is sized incorrectly here:

this.buffer = ByteBuffer.allocate((int) Math.ceil(encoder.maxBytesPerChar()));

It appears often encoders try to encode supplementary code points (represented by a surrogate pair consisting of two char) either completely or not at all, so the buffer with size encoder.maxBytesPerChar() is too small, using 2 * encoder.maxBytesPerChar() should solve this issue.

However, even when this is fixed, it would be good to improve the encoding logic by checking for OVERFLOW (instead of silently ignoring it) and throwing an AssertionError here:

CoderResult result = encoder.encode(readBuffer, output, false);

Activity

  1. added a commit that references this issue on Oct 14, 2021
    310f5df
  2. added a commit that references this issue on Jun 2, 2024
    deb7469
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions