Visitar URL original
fix(kafka sink): send no key when key_field is null by anpawo · Pull Request #27145 · vectordotdev/vector · GitHub
Skip to content

fix(kafka sink): send no key when key_field is null - #27145

Open
anpawo wants to merge 1 commit into
vectordotdev:masterfrom
anpawo:fix/kafka-null-key
Open

anpawo wants to merge 1 commit into
vectordotdev:masterfrom
anpawo:fix/kafka-null-key

Conversation

@anpawo

@anpawo anpawo commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Summary

With key_field set, an event whose key field is explicitly null was produced with the literal string <null> as its Kafka key, because get_key coerced the value to bytes whatever its type. An event without the field was already produced with no key.

get_key now treats a null value like a missing one for log and trace events. Metric tags need no change: MetricTags::get already skips null values.

References

Issue: #24728

Vector configuration

sinks:
  kafka_out:
    type: kafka
    inputs: [decode_message]
    bootstrap_servers: localhost:9092
    topic: test
    key_field: key
    encoding:
      codec: json

Input {"test": "test", "key": null} now produces a message with no key, like {"test": "test"}.

How did you test this PR?

Added sinks::kafka::request_builder::tests::kafka_get_key_null_is_no_key, which checks that a null key gives no key for log and trace events and that a string key is unchanged. It fails without the change (Some(b"<null>")). The Kafka integration tests were not run.

cargo test --no-default-features --features sinks-kafka --lib sinks::kafka::request_builder

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
    For a new deprecation announcement, follow the deprecation policy and add a deprecation.d/ notice instead.
  • No. A maintainer will apply the no-changelog label to this PR.

Contributor Guidelines

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Before pushing, follow our pre-push guidance.
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.

This PR was written with Claude Opus 5.5 as a coding agent and reviewed with Codex.

Closes: #24728

`get_key` coerced the `key_field` value to bytes whatever its type, and
`Value::Null` coerces to the string "<null>". An event with an explicit
`key: null` was therefore produced with the literal key "<null>", while
an event without the field was produced with no key, as users expect.

Treat a null value like a missing one for log and trace events.

Closes vectordotdev#24728
@anpawo
anpawo requested a review from a team as a code owner November 12, 2026 13:31
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T13:32:55.560407Z 3b27e8a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions github-actions Bot added the domain:sinks Anything related to Vector's sinks label Nov 12, 2026

@datadoghq-integration datadoghq-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Null log and trace keys now use Kafka’s existing absent-key handling, avoiding the literal <null> key while preserving non-null key conversion.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 3b27e8a

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain:sinks Anything related to Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect key "<null>" in kafka sink when key_field is null

1 participant