Describe the bug
array_min / array_max on a zero-row List<Utf8> (or any non-primitive element type) fails:
Execution error: Empty iterator passed to ScalarValue::iter_to_array
A zero-row List<Int64> works. array_min_max_helper (datafusion/functions-nested/src/min_max.rs) only has a fast path for primitive types; the fallback builds a Vec<ScalarValue> and calls ScalarValue::iter_to_array, which rejects an empty iterator.
Zero-row batches do reach these functions in practice: ProjectionExec and TopK evaluate expressions on every batch, so ORDER BY array_min(tags) LIMIT n fails whenever an input emits an empty batch.
To Reproduce
use std::sync::Arc;
use datafusion::arrow::array::{ArrayRef, ListBuilder, StringBuilder};
use datafusion::arrow::record_batch::RecordBatch;
use datafusion::prelude::SessionContext;
#[tokio::main]
async fn main() -> datafusion::error::Result<()> {
let ctx = SessionContext::new();
let tags: ArrayRef = Arc::new(ListBuilder::new(StringBuilder::new()).finish());
ctx.register_batch("t", RecordBatch::try_from_iter(vec![("tags", tags)])?)?;
ctx.sql("SELECT array_min(tags) FROM t").await?.show().await?; // error
Ok(())
}
Seen on 54.0.0; the code is unchanged on main.
Expected behavior
Zero rows, the same as for primitive element types.
Additional context
Found in OpenSearch's DataFusion backend when sorting on multi-valued keyword fields with an empty shard. Proposed fix: return new_empty_array(&array.value_type()) for an empty input before the fallback. PR to follow.
Describe the bug
array_min/array_maxon a zero-rowList<Utf8>(or any non-primitive element type) fails:A zero-row
List<Int64>works.array_min_max_helper(datafusion/functions-nested/src/min_max.rs) only has a fast path for primitive types; the fallback builds aVec<ScalarValue>and callsScalarValue::iter_to_array, which rejects an empty iterator.Zero-row batches do reach these functions in practice:
ProjectionExecandTopKevaluate expressions on every batch, soORDER BY array_min(tags) LIMIT nfails whenever an input emits an empty batch.To Reproduce
Seen on 54.0.0; the code is unchanged on
main.Expected behavior
Zero rows, the same as for primitive element types.
Additional context
Found in OpenSearch's DataFusion backend when sorting on multi-valued keyword fields with an empty shard. Proposed fix: return
new_empty_array(&array.value_type())for an empty input before the fallback. PR to follow.