[NativeAOT] Array marshalling differences exposed by pointer-array tests

[jkoritzinsky]

The pointer-array tests enabled for NativeAOT in #134284 expose three existing marshalling differences. This issue tracks the runtime fixes and re-enabling the narrowly excluded cases; the pointer-array PR will not change NativeAOT implementation code.

Observed failures

Build 1611133, commit 0c9d8ed20049c5544988c9ad301c3aa106292a96 (CI merge 6c596a6378674dddf2886805ba2a1810275d400c), Windows x64 NativeAOT Checked. The test-run display name says coreclr windows x64 Checked, but the Interop console uses nativeaottest.cmd. The uploaded result XML contains 27 failing cases in these four test methods:

Cases	Actual behavior	Expected behavior
PointerArrayFieldTests.CopyFixedArrays(true) (1 case)	A null ByValArray field leaves prefilled native memory unchanged (0x123456789ABCDEF0).	Clear the field's native array storage.
PointerArrayTests.CopyArrayByRef(kind, -1) and CopyOutArray(kind, -1) (14 + 4 cases)	A null native array with length zero becomes an empty managed array.	Preserve null.
PointerArrayTests.CopyArrayInReversePInvoke(functionPointers, length) for lengths 1, 4, 5, and 21 (8 cases)	The native verification returns 0 after the managed callback reverses the array.	Copy the callback's pointer values back to the original native buffer.

The corresponding non-null forward-copy, pinning, and fixed-field cases pass. These tests also pass on CoreCLR. The failures were identified from CI results and source inspection; they have not been independently reproduced with a local NativeAOT execution.

Source inspection

• ByValArrayMarshaller.EmitMarshalFieldManagedToNative branches to the end when the managed field is null, without clearing its native storage.
• ArrayMarshaller.TransformNativeToManaged allocates an array for byref parameters based on the element count without checking whether the native pointer is null.
• Reverse array copy-out selects an element marshaler with In = false in GetElementMarshaller. EmitMarshalElementManagedToNative then calls AllocAndTransformManagedToNative, which skips the value transfer when In is false. For blittable pointer elements, this emits default native values rather than the managed values.

Re-enable condition

After correcting these paths, re-enable the NativeAOT exclusions associated with this issue and run the exact null-array, null-fixed-field, and nonempty reverse-copy cases, while retaining the currently passing array coverage.

Note

This issue was prepared with GitHub Copilot assistance from CI evidence and source inspection.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.