Repository navigation
Conversation
Signed-off-by: 1fanwang <1fannnw@gmail.com>
…sync-stream-text Signed-off-by: 1fanwang <1fannnw@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
At current main, commit be49830, Vertex AI async streaming calls can finish with missing or incorrect response text in the recorded span. The streaming wrapper does accumulate the full text while it drains the stream, but the code that finalizes the span gets the exhausted generator object instead of the text that was just collected.
Fix
Pass the accumulated async stream text into the response-finalization path once the drain completes, and add a regression test around the real async streaming wrapper so this does not regress.
Testing Done
The red and green probe uses a local async generator substitute rather than a live Vertex AI account. This bug is pure response-plumbing logic inside the instrumentation layer, so the substitute is enough to drive the real code path honestly. The substitute yields chunk objects with the same fields this path reads from the SDK.
Red
Command:
cd packages/opentelemetry-instrumentation-vertexai PYTHONWARNINGS=ignore::FutureWarning uv run python /tmp/vertexai_async_stream_probe.pyOutput before this patch:
Green
Command:
cd packages/opentelemetry-instrumentation-vertexai PYTHONWARNINGS=ignore::FutureWarning uv run python /tmp/vertexai_async_stream_probe.pyOutput with this patch:
Reproducer source: vertexai_async_stream_probe.py