Skip to content

Promote database queue reliability fixes and enqueue improvements - #38

Merged
elpete merged 9 commits into
mainfrom
next
Sep 9, 2026
Merged

Promote database queue reliability fixes and enqueue improvements#38
elpete merged 9 commits into
mainfrom
next

Conversation

@elpete

@elpete elpete commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Promote the database queue fixes and enqueue improvements from next to main.

Database callbacks are fenced to their execution attempt, retry counts support long-lived jobs, and unknown job mappings remain available for a compatible worker. Batch results are counted once per job ID so duplicate callbacks cannot decrement pendingJobs below zero or repeat lifecycle callbacks.

Database connections gain a configurable polling interval and opt-in batched enqueueing. Queue names and payloads are bound as text. Documentation covers MySQL compatibility, the bundled UUID library, worker routing, and upgrade requirements.

Upgrade: run migrations 000010 (widen job attempts) and 000011 (track processed batch jobs) before deploying, including equivalent changes for custom tables. Drain pre-existing batches if successful callbacks may be redelivered: historical successful job IDs cannot be reconstructed. The batch accounting commit (7db843d) places BREAKING CHANGE: in the third paragraph, as required by commandbox-semantic-release 3.0.1. Its actual parser and analyzer were exercised with the corrected message and returned isBreakingChange=true and releaseType=major.

Validation: constituent PR checks passed on Lucee 5/6 and Adobe 2021/2023/2025. This promotion will be merged only after its current head passes all checks.

* fix: fence database callbacks to their execution attempt

* test: align direct marshal fixture with persisted attempt

* test: avoid literal array iteration on older Adobe engines
* fix: widen persisted job execution counter

* fix: bind persisted execution counts as bigint

* refactor: use unprefixed bigint parameter types
* perf(db): bind queue names and job payloads as text

* test(db): compare queue names as literal text

* fix(db): omit cf_sql prefix from text bindings
* feat(db): support opt-in batched job enqueueing

* test(db): check numeric payload keys across engines
Record processed job IDs under the batch row lock, ignore repeated outcomes, and suppress duplicate lifecycle callbacks. Cover duplicate, concurrent, exhausted, and pre-migration batch cases. Refs #19.

BREAKING CHANGE: Run migrations 2000_01_01_000010_widen_job_attempts and 2000_01_01_000011_track_processed_batch_jobs before deploying this version, including equivalent changes for custom tables. Drain pre-existing batches if successful job callbacks may be redelivered, since historical successful IDs cannot be reconstructed.
Release only the fetched database reservation without consuming an attempt, log the worker and mapping, and continue polling. Add regression coverage for missing mappings, datasource isolation, explicit dispatch connections, and optional batch callbacks.

Refs #6, #5, #12
Document MySQL 8 SKIP LOCKED requirements, the bundled UUID Java load path, worker queue routing, and the batch accounting migration.

Refs #10, #13, #19, #6
@elpete
elpete merged commit 95b54b2 into main Sep 9, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant