Skip to content

[IE] Fix depth kernel index in 3D TransposedConvolution unroll - #419

Open
dyang23 wants to merge 2 commits into
openvinotoolkit:developfrom
dyang23:fix/unroll-3d-tconv-depth-flip
Open

dyang23 wants to merge 2 commits into
openvinotoolkit:developfrom
dyang23:fix/unroll-3d-tconv-depth-flip

Conversation

@dyang23

@dyang23 dyang23 commented Nov 17, 2026

Copy link
Copy Markdown

Summary

UnrollConv3dToConv2d (TransposedConvGeneralRewriter) unrolls a 3D IE.TransposedConvolution into 2D TransposedConvolutions,
one per (input depth, kernel depth) pair, and places each result at output depth actIndex * strideZ + depthIndex - padBeginZ.

The IE.TransposedConvolution filter is already spatially reversed: LegalizeConvBackpropData reverses Z/Y/X when it converts
ConvolutionBackpropData. Y/X are handled correctly because the 2D TransposedConvolution consumes the reversed kernel. In depth,
though, the pass takes slicedFilters[depthIndex] from the reversed filter while placing the result with the original
(non-reversed) index. As a result, the kernel is applied flipped along depth.

Effect: every 3D ConvTranspose / ConvolutionBackpropData with kernel depth > 1 returns wrong values on the NPU, with no error or
warning. Kernel depth 1 and 2D deconvolution are not affected.

Fix: use slicedFilters[kernelZ - 1 - depthIndex].

Commits:

  1. The fix, plus updated expectations in the existing 37XX/40XX TransposedConvolution 3D tests. Those tests encoded the old
    mapping; it went unnoticed because they use kernelZ = 2 and mostly splat filters.
  2. New non-splat regression tests (kernelZ = 3, a different value in every depth slice) for 37XX/40XX and for 50XX+. The
    50XX+ file had no TransposedConvolution coverage for this pass.

JIRA ticket

  • E#243504

Target Platform For Release Notes

  • NPU37XX
  • NPU40XX
  • NPU50XX
  • NPU60XX
  • NONE (Not included in release notes)

Classification of this Pull Request

  • Maintenance
  • BUG
  • Feature

Validation

LIT (Release build on top of develop 56e4d44, OpenVINO 88f8c6bf):

  • unroll_conv3d_to_conv2d_37XX_40XX.mlir and unroll_conv3d_to_conv2d_50XX+.mlir pass on NPU3720, NPU4000 and NPU5010.
  • With the fix reverted, the new tests fail on NPU4000 and NPU5010, so they detect the bug.
  • Whole dialect/IE directory: 4 failures per platform, and the same 4 also fail without this change (unrelated to this PR):
    convert_precision_to_i32, dump_statistics_of_ie_ops, constant_tracing_ws, low_precision_asymmetric_zp_per_tensor_true_37XX_40XX_50XX_60XX.

On device: PTL (NPU5010), Linux NPU driver 1.32.0.20260402, OpenVINO 2026.4.1. The CiD library was built with the
job_build_cid.yml options and loaded via LD_LIBRARY_PATH, with a fresh driver cache for every run. The fixed and unfixed
builds differ only by this change.

3D ConvTranspose, cos vs CPU f32 unfixed fixed
k4 s2 p1 (1x48x6x15x20) 0.004 1.00000
k4 s2 p1 (1x16x24x60x80) 0.013 1.00004
k2 s2 p0 0.027 1.00000
k3 s2 p1 0.398 1.00000
k3 s1 p1 0.379 1.00000
k1 s1 p0 / 2D control 1.00000 1.00000

The unfixed build reproduces the installed driver's results exactly.

End-to-end, IGEV++ stereo (3 x ConvTranspose3d k4 s2 p1), Middlebury MiddEval3 15 scenes, 480x640, unmodified model:

  • NPU EPE goes from 3.30 px to 0.89 px (CPU f32: 0.89).
  • Pixels off by > 3 px vs CPU go from 33% to 0.03%.
  • Latency is unchanged (169.5 vs 169.3 ms).

Related PRs

  • None

🤖 Generated with Claude Code

@dyang23
dyang23 requested a review from a team as a code owner November 17, 2026 02:26
dyang23 and others added 2 commits November 17, 2026 10:53
UnrollConv3dToConv2d splits a 3D TransposedConvolution into 2D
TransposedConvolutions per (input depth, kernel depth) pair and places
each result at output depth actIndex * strideZ + depthIndex.

The IE.TransposedConvolution filter is stored spatially reversed in Z, Y
and X (LegalizeConvBackpropData reverses all spatial axes). Y/X are
handled correctly by the 2D TransposedConvolution, but the Z slice was
taken as slicedFilters[depthIndex], i.e. from the reversed filter, while
the placement uses the original kernel index. Net effect: the kernel is
applied flipped along depth, so any 3D ConvolutionBackpropData /
ConvTranspose3d with kernelZ > 1 returns wrong results on NPU (cosine
similarity ~0 vs CPU for k4 s2 p1; kernelZ == 1 is unaffected).

Use slicedFilters[kernelZ - 1 - depthIndex] and update the LIT
expectations accordingly.

Observed on PTL NPU (driver 1.32.0.20260402, OV 2026.3/2026.4.1):
the NPU output of a single ConvolutionBackpropData 5D op equals the CPU
reference computed with the kernel flipped along D (cos 1.00000).
IGEV++ stereo depth (3x ConvTranspose3d k4 s2 p1 in cost aggregation)
gives Middlebury EPE 3.30 px on NPU vs 0.89 px on CPU; with the weights
pre-flipped along D the NPU result is 0.892 px.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…roll

The existing 3D TransposedConvolution unroll tests use kernelZ = 2 and mostly
splat (all-ones) filters, so the per-depth filter slices are identical and a
wrong depth kernel index cannot be detected.

Add a kernelZ = 3 case whose filter holds different values in every depth
slice, and check which slice each unrolled 2D TransposedConvolution uses,
which input depth it reads, and the order of the concatenated output.
The case is added for NPU37XX/NPU40XX and for NPU50XX+ (which had no
TransposedConvolution coverage in this pass).

Both new cases fail without the preceding fix and pass with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dyang23
dyang23 force-pushed the fix/unroll-3d-tconv-depth-flip branch from e88be89 to 625b0a5 Compare November 17, 2026 02:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant