Visitar URL original
Spark: Fix NPE in insert op for ORC format with variant type in v3 tables by akashrn5 · Pull Request #18387 · apache/iceberg · GitHub
Skip to content

Spark: Fix NPE in insert op for ORC format with variant type in v3 tables - #18387

Open
akashrn5 wants to merge 1 commit into
apache:mainfrom
akashrn5:NPE
Open

Spark: Fix NPE in insert op for ORC format with variant type in v3 tables#18387
akashrn5 wants to merge 1 commit into
apache:mainfrom
akashrn5:NPE

Conversation

@akashrn5

@akashrn5 akashrn5 commented Oct 5, 2026 •
edited
Loading

Copy link
Copy Markdown

Closes #18386

Summary

Writing a VARIANT column to a v3 table whose write format is ORC fails with a
NullPointerException from inside SparkOrcWriter, instead of the
UnsupportedOperationException("Variant is not supported") that
OrcSchemaWithTypeVisitor already defines for this case.

@github-actions github-actions Bot added the spark label Oct 5, 2026
@akashrn5 akashrn5 changed the title [WIP]Fix NPE in insert op for ORC format with variant type in v3 tables [WIP]Spark: Fix NPE in insert op for ORC format with variant type in v3 tables Oct 5, 2026
@akashrn5 akashrn5 changed the title [WIP]Spark: Fix NPE in insert op for ORC format with variant type in v3 tables Spark: Fix NPE in insert op for ORC format with variant type in v3 tables Oct 12, 2026
@akashrn5

akashrn5 commented Oct 12, 2026

Copy link
Copy Markdown
Author

Hi @nastra @uros-b can anyone please have a look at this PR? or if you can ask anyone else to review this. Thanks

nastra
nastra reviewed Oct 12, 2026
View reviewed changes
Comment thread spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestOrcWrite.java
}

@Test
public void variantIsNotSupported() throws IOException {

@nastra nastra Oct 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this only covers the write path. Can we also have a test for the two read paths that are modified?

@akashrn5 akashrn5 Oct 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added the test case @nastra

nastra
nastra reviewed Oct 12, 2026
View reviewed changes
Comment thread .../spark/src/main/java/org/apache/iceberg/spark/data/vectorized/VectorizedSparkOrcReaders.java Outdated
nastra
nastra reviewed Oct 12, 2026
View reviewed changes
Comment thread spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/data/TestOrcWrite.java Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nastra nastra nastra left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

spark

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Spark/ORC] NullPointerException in SparkOrcWriter when inserting VARIANT data into ORC-backed Iceberg v3 tables

2 participants

@akashrn5 @nastra