Describe the bug, including details regarding any error messages, version, and platform.
ListArray::FromListView (and LargeListArray::FromListView) returns the wrong validity for a sliced list-view array with nulls.
auto views = arrow::json::ArrayFromJSONString(
arrow::list_view(arrow::int32()),
"[[1], [2], [3], [4], [5], [6], [7], [8], null, [10], [11]]")
.ValueOrDie();
auto sliced = std::static_pointer_cast<arrow::ListViewArray>(views->Slice(1));
auto lists =
arrow::ListArray::FromListView(*sliced, arrow::default_memory_pool()).ValueOrDie();
// is null (list view): 0 0 0 0 0 0 0 1 0 0
// is null (list): 0 0 1 1 1 1 1 1 1 1
Expected: the same nulls as the input.
In ListFromListViewImpl (cpp/src/arrow/array/array_nested.cc), the validity bitmap is read with list_view_data->GetValues<uint8_t>(0), which already advances the pointer by offset bytes. It is then passed to bit_util::GetBit(in_validity_bitmap, list_view_data->offset + i), which applies the offset again, in bits. FlattenListViewArray in the same file does this correctly with GetValues<uint8_t>(0, 0).
Found while trying to use FromListView in a compute kernel (GH-33295). Related: GH-51612, where the list_view to list cast could use FromListView. Reproduced on current main, Linux x86_64.
Component(s)
C++
Describe the bug, including details regarding any error messages, version, and platform.
ListArray::FromListView(andLargeListArray::FromListView) returns the wrong validity for a sliced list-view array with nulls.Expected: the same nulls as the input.
In
ListFromListViewImpl(cpp/src/arrow/array/array_nested.cc), the validity bitmap is read withlist_view_data->GetValues<uint8_t>(0), which already advances the pointer byoffsetbytes. It is then passed tobit_util::GetBit(in_validity_bitmap, list_view_data->offset + i), which applies the offset again, in bits.FlattenListViewArrayin the same file does this correctly withGetValues<uint8_t>(0, 0).Found while trying to use
FromListViewin a compute kernel (GH-33295). Related: GH-51612, where thelist_viewtolistcast could useFromListView. Reproduced on currentmain, Linux x86_64.Component(s)
C++