[Python][C++] Casting list_view to list ignores the view sizes and gives wrong results

[jonasdedden]

Describe the bug, including details regarding any error messages, version, and platform.

Casting a list_view array to list (or large_list_view to large_list, or across offset widths) returns wrong results whenever the views aren't laid out like list offsets, e.g. when they're out of order or overlap.

import pyarrow as pa

views = pa.ListViewArray.from_arrays(
    pa.array([2, 0, 1], pa.int32()),  # offsets
    pa.array([2, 2, 1], pa.int32()),  # sizes
    pa.array([1, 2, 3, 4]),
)
print(views.to_pylist())                        # [[3, 4], [1, 2], [2]]
print(views.cast(pa.list_(pa.int64())).to_pylist())  # [[], [1], []]

Expected: [[3, 4], [1, 2], [2]].

The cast kernels are registered with the generic list-to-list cast (AddListCast<ListViewType, ListType> and friends in cpp/src/arrow/compute/kernels/scalar_cast_nested.cc, added in GH-39051). CastList::Exec reuses the input's buffer 1 as the output's list offsets. For a list view, buffer 1 holds the per-view offsets, and the sizes buffer is ignored, so the result is only right when every offsets[i] + sizes[i] == offsets[i + 1].

ListArray::FromListView / LargeListArray::FromListView already copy the referenced values into a proper list layout, so the cast kernel could use that (but see GH-51613: FromListView currently gets the nulls of sliced inputs wrong).

Reproduced on current main (C++ and a PyArrow dev build), Linux x86_64.

Component(s)

C++, Python

[rajaryan2007]

I'm working on this one , just to inform .