Skip to content

fix(templates): Pin Playwright to base image version for templates#1996

Merged
Pijukatel merged 5 commits into
masterfrom
claude/playwright-version-mismatch-9ewoa8
Jul 1, 2026
Merged

fix(templates): Pin Playwright to base image version for templates#1996
Pijukatel merged 5 commits into
masterfrom
claude/playwright-version-mismatch-9ewoa8

Conversation

@Pijukatel

@Pijukatel Pijukatel commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

  • Use fixed playwright_version to determine the version of the base image and also the version of installed Playwright

@claude
fix: Pin playwright to base image version for pip and poetry templates
c4ebd6a 
The project template Dockerfile relied on a `sed` substitution to align
the installed playwright version with the browser binaries baked into the
`apify/actor-python-playwright*` base images. That substitution only
rewrites a line starting with `playwright==`, but neither `requirements.txt`
(pip) nor the exported `pyproject.toml`/`poetry.lock` reliably pin playwright
directly — it is pulled in transitively via `crawlee[playwright]`. As a
result pip installed the latest playwright release, replacing the version
that matches the base image's browser binaries and causing a protocol
mismatch (e.g. `Browser.setDefaultViewport` errors with camoufox).

Replace the fragile `sed` trick in the pip and poetry branches with the
same explicit pin already used by the uv branch: snapshot the base image's
pre-installed playwright version, then force-reinstall that exact version
after dependency installation. Non-playwright base images have no playwright
pre-installed, so the pin is skipped there.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012xSK5xwEhCUXbAeC9SVf29
@codecov

codecov Bot commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.34%. Comparing base (b5f0cb4) to head (e6080d3).
⚠️ Report is 10 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #1996      +/-   ##
==========================================
+ Coverage   93.08%   93.34%   +0.25%     
==========================================
  Files         167      179      +12     
  Lines       11770    12482     +712     
==========================================
+ Hits        10956    11651     +695     
- Misses        814      831      +17
Flag Coverage Δ
unit 93.34% <ø> (+0.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Pijukatel Pijukatel marked this pull request as draft June 30, 2026 07:05
@claude
fix: Snapshot base playwright version before dependency install for p…
3459e14 
…ip/poetry

The previous attempt captured the base image's playwright version inside the
dependency-install RUN. But the e2e template test patches the generated
project (tests/e2e/project_template/utils.py) by injecting a
`pip install ./crawlee-*.whl[playwright,...] --force-reinstall` step right
after `COPY requirements.txt ./` / `COPY pyproject.toml ...`. That injected
step resolves the unpinned playwright to the latest release *before* the
snapshot ran, so the version captured (and pinned to) was the already-upgraded
one — leaving the camoufox browser binary protocol mismatch unfixed.

Move the snapshot into its own RUN placed before the COPY (and thus before any
injected install step), writing it to /tmp/.base-playwright-version and reading
it back when pinning — mirroring exactly what the working uv branch already
does. This guarantees the captured version is the one baked into the base image
and matching its browser binaries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012xSK5xwEhCUXbAeC9SVf29
@Pijukatel

Pijukatel commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator Author

E2E tests: https://github.com/apify/crawlee-python/actions/runs/28428916800

@Pijukatel Pijukatel requested a review from vdusek June 30, 2026 08:08
@Pijukatel Pijukatel marked this pull request as ready for review June 30, 2026 08:08
@B4nan

B4nan commented Jun 30, 2026

Copy link
Copy Markdown
Member

I see we still have this python specific setup in the docker images. We should align this, in JS we have docker image tags for playwright versions, and we pin those in the templates (together with the dependency version, to ensure they are aligned, plus we have a warning in the docker build if those versions are not the same).

cc @vladfrangu

it is pulled in transitively via crawlee[playwright]

This is also weird, in JS, we don't dictate the playwright version, its a peer dependency, users need to install it explicitly. Can we do the same in python?

@Pijukatel Pijukatel mentioned this pull request Jun 30, 2026
Merged
@vdusek vdusek changed the title fix: Pin playwright to base image version for pip and poetry templates fix(templates): Pin playwright to base image version for pip and poetry templates Jun 30, 2026
Pijukatel added a commit that referenced this pull request Jun 30, 2026
@Pijukatel
fix(templates): Use Playwright version locked base docker images (#1997)
b5f0cb4 
### Description

Base Docker images are bumping Playwright version automatically. This
can make templates broken. Bumping the version in templates should be a
manual step.

(Should be used together with
#1996 for fixing and
stabilizing templates)

### Issues

<!-- If applicable, reference any related GitHub issues -->

- Closes: [295](apify/apify-actor-docker#295)
vdusek
vdusek reviewed Jun 30, 2026
View reviewed changes

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should merge #1997 first. After that, couldn't we use a build ARG or ENV instead? I don't think we need to derive the Playwright version this way - we should already know it.

@Pijukatel Pijukatel marked this pull request as draft June 30, 2026 10:00
@Pijukatel

Pijukatel commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator Author

@vdusek We can fix the version in the template, for example, like this c25bd14

There are some edge cases about users who want a specific Playwright version or specific browser versions, but for that they have to do some custom Dockerfile changes.

E2E tests green: https://github.com/apify/crawlee-python/actions/runs/28436119038

@Pijukatel Pijukatel mentioned this pull request Jun 30, 2026
Merged
@Pijukatel Pijukatel marked this pull request as ready for review June 30, 2026 11:46
@Pijukatel Pijukatel requested a review from vdusek June 30, 2026 11:46
@Pijukatel Pijukatel changed the title fix(templates): Pin playwright to base image version for pip and poetry templates fix(templates): Pin Playwright to base image version for templates Jun 30, 2026
vdusek
vdusek reviewed Jul 1, 2026
View reviewed changes

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions / suggestions

Comment thread src/crawlee/project_template/{{cookiecutter.project_name}}/Dockerfile Outdated
Comment thread src/crawlee/project_template/{{cookiecutter.project_name}}/Dockerfile Outdated
Comment thread src/crawlee/project_template/{{cookiecutter.project_name}}/Dockerfile
@Pijukatel

Pijukatel commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

New run of E2E tests with the latest commit:
https://github.com/apify/crawlee-python/actions/runs/28504970808

@Pijukatel Pijukatel requested a review from vdusek July 1, 2026 09:18
@Pijukatel Pijukatel merged commit 917b6b6 into master Jul 1, 2026
128 checks passed
@Pijukatel Pijukatel deleted the claude/playwright-version-mismatch-9ewoa8 branch July 1, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vdusek vdusek vdusek approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Pijukatel @B4nan @vdusek @apify-service-account @claude