[SPARK-57271][PYTHON] Propagate traceback locals to Python planner runner

[linhongliu-db]

What changes were proposed in this pull request?

This PR propagates SPARK_TRACEBACK_WITH_LOCALS from PythonPlannerRunner when spark.sql.execution.pyspark.udf.tracebackWithLocals.enabled is enabled.

It also adds a UDTF analyze regression test that raises from the planner-side Python worker and verifies the surfaced traceback includes the local variable.

Why are the changes needed?

PythonPlannerRunner already reads spark.sql.execution.pyspark.udf.tracebackWithLocals.enabled, but it did not add SPARK_TRACEBACK_WITH_LOCALS to the Python worker environment. As a result, planner-driven Python paths such as UDTF analyze did not honor the traceback-locals config, unlike regular Python UDF execution.

Does this PR introduce any user-facing change?

Yes. When spark.sql.execution.pyspark.udf.tracebackWithLocals.enabled is enabled, Python exceptions raised through planner-driven Python paths can now include local variables in their tracebacks.

How was this patch tested?

Passed:

python3 -m py_compile python/pyspark/sql/tests/test_udtf.py
python3 -m py_compile python/pyspark/sql/tests/test_udf.py
git diff --check

Attempted but not run locally because the Spark assembly could not be built in this environment:

./build/sbt assembly/package
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 ./build/mvn -DskipTests -pl :spark-assembly_2.13 -am package
python3 python/run-tests.py --testnames 'pyspark.sql.tests.test_udtf UDTFTests.test_udtf_analyze_traceback_with_locals'
python3 python/run-tests.py --testnames 'pyspark.sql.tests.test_udf UDFTests.test_udf_traceback_with_locals'

The SBT launcher fetch failed, and Maven dependency resolution failed because Maven Central hostnames did not resolve from this local environment. The PySpark test runner then failed at startup with Cannot find assembly build directory, please build Spark first.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex (GPT-5)