Skip to content

Fix: render execution_time through a per-test generator instead of patching the shared dialect - #6031

Open
niklasdohmen wants to merge 1 commit into
SQLMesh:mainfrom
niklasdohmen:fix/thread-safe-execution-time-rendering
Open

Fix: render execution_time through a per-test generator instead of patching the shared dialect#6031
niklasdohmen wants to merge 1 commit into
SQLMesh:mainfrom
niklasdohmen:fix/thread-safe-execution-time-rendering

Conversation

@niklasdohmen

@niklasdohmen niklasdohmen commented Sep 7, 2026

Copy link
Copy Markdown

Problem

ModelTest implements execution_time by patching two process-global dictionaries on the dialect's generator class — TRANSFORMS and SQLGlot's cached dispatch table — inside _concurrent_render_context. The lock there covers only the model and CTE renders; setUp (fixture CREATE VIEW) and tearDown render SQL outside it.

With concurrent_tasks > 1 and at least one test that sets execution_time, another thread can hit the window in which patch.dict restores the dispatch table (clear, then refill) and fail with

ValueError: Unsupported expression type Create

while creating its fixture views. Less visibly, any other thread that renders CURRENT_DATE/CURRENT_TIMESTAMP during the patched window gets the first test's frozen time. We saw the crash on a 106-test Databricks project running 16-wide and had to serialise the whole suite (15 minutes instead of about 1).

Fix

  • Build the frozen-time transforms into a per-test generator subclass (_frozen_time_generator_class, cached per generator/time/dialect) instead of patching the shared class. SQLGlot caches one dispatch table per generator class, so the subclass gets its own and the shared one is never touched.
  • SQL model tests render the model query and the CTE queries through that subclass (_generate_sql). _concurrent_render_context keeps the lock only for what is genuinely process-global: time_machine.travel.
  • Python model tests can run arbitrary SQL through the engine adapter, whose generator cannot be swapped per test, so they keep the previous shared-dialect patch under the lock (patch_shared_dialect=True). That path is unchanged in behaviour.

Test

test_freeze_time_does_not_patch_shared_generator: inside a frozen test's render context, another thread renders CREATE VIEW and CURRENT_DATE through the shared duckdb dialect 200 times; asserts no error, that the shared dialect still renders CURRENT_DATE, that the test's own generator renders the frozen cast, and that the shared TRANSFORMS and dispatch table are byte-identical afterwards. On main the shared dialect renders the frozen cast inside the context, which this test rejects; passes with this change. tests/core/test_test.py otherwise unchanged in outcome.

… patch

ModelTest froze time by patching the dialect's generator TRANSFORMS and
SQLGlot's cached dispatch table, which are process-global, under a lock
that only covers the model and CTE renders. setUp (fixture CREATE VIEW)
and tearDown render outside that lock, so with concurrent_tasks > 1
another test could observe the dispatch table mid-restore and fail with
"Unsupported expression type Create", or silently render CURRENT_* at the
other test's frozen time.

Build the frozen-time transforms into a per-test generator subclass and
render SQL model tests through it; keep the lock for time_machine only.
Python model tests still patch the shared dialect under the lock because
their SQL goes through the engine adapter.

Signed-off-by: Niklas Dohmen <niklas@enam.co>
@niklasdohmen
niklasdohmen force-pushed the fix/thread-safe-execution-time-rendering branch from 15ba5cc to 55d2c86 Compare September 7, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant