fix: the config file's directory no longer decides the rootdir alone - #14837
Open
RonnyPfannschmidt wants to merge 3 commits into
Open
fix: the config file's directory no longer decides the rootdir alone#14837RonnyPfannschmidt wants to merge 3 commits into
RonnyPfannschmidt wants to merge 3 commits into
Conversation
Knowledge about config files was spread across three places that had to be kept in sync by hand: load_config_dict_from_file dispatched on suffix with filename checks nested inside those branches, locate_config re-declared the discovery order in its own hardcoded list, and adding a format meant editing both in the right order. Introduce two tables instead. CONFIG_LOADERS maps a config file *name* to its loader and doubles as the discovery order used by locate_config, so order and parsing can no longer drift apart. CONFIG_SUFFIXES maps a *suffix* to a loader for files passed explicitly via -c/--config-file, which may be named anything. load_config_dict_from_file now looks up the name first and falls back to the suffix. The per-format rules move out of nested conditionals into named functions -- _parse_pytest_ini, _parse_ini_file, _parse_cfg_file, _parse_pytest_toml and _parse_pyproject_toml -- each documenting the rule it implements. This is a pure refactor: no test needed changing, and running load_config_dict_from_file over a matrix of every supported name crossed with present/absent/empty/malformed sections (plus unsupported and extension-less files) yields identical values, modes, origins and exceptions before and after. Two pre-existing warts are deliberately preserved rather than fixed here: the CFG_PYTEST_SECTION message still names setup.cfg for any .cfg file, and a scalar `pytest` key still raises AttributeError. This redoes the structural half of pytest-dev#8358 on top of current main; that PR bundled it with the abandoned setup.cfg deprecation from pytest-dev#3523. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Consolidates three open reports that all bottom out in how an explicitly given config file is located and parsed. The preceding refactor turns each of them into a small change rather than another special case in a conditional. Custom TOML files also read [pytest] (pytest-dev#14705) pytest documents [pytest] as the table its own TOML configuration files use, but a TOML file passed via -c was parsed with pyproject.toml semantics, so a [pytest] table in it was silently ignored. Such files now read [pytest] as well, while the [tool.pytest]/[tool.pytest.ini_options] tables they were previously restricted to keep working -- writing both styles into one file is a UsageError. Suffix dispatch makes this one loader; pyproject.toml itself is unaffected, as name dispatch wins. -c/--config-file validates its argument (pytest-dev#14716) Previously an invalid path either silently produced an empty configuration -- while still reporting `configfile:` in the header -- or crashed with a raw FileNotFoundError traceback, depending on its extension. Now a path that does not exist, a directory, and a regular file pytest has no loader for are each a UsageError. The supported extensions in the message are derived from CONFIG_LOADERS_BY_SUFFIX rather than restated in a separate constant. This is breaking for invocations that passed an unparsable file to -c and relied on it being ignored. The pytest-dev#14683 regression test did exactly that with a conftest.py and now uses a real config file; it passes --rootdir explicitly, so the config file was incidental to what it covers. The rootdir is not derived from a non-regular config file (pytest-dev#11502) --config-file=/dev/null is a common way to load no configuration at all. Deriving the rootdir from its parent made the rootdir /dev, and the cache plugin then warned on every run that it could not create /dev/.pytest_cache. Such a path says nothing about where the project lives, so fall back to the usual common-ancestor logic. Whether a path is parsed and whether its directory decides the rootdir are kept separate: a loader runs whenever one matches the name or suffix, so a config file that happens to be a fifo is still read, and only a path with no loader *and* no chance of holding configuration -- a character device such as /dev/null -- means "no configuration" instead of an error. The diagnoses come from pytest-dev#14707 (@DebadityaHait), pytest-dev#14723 (@wanxiankai) and pytest-dev#14671 (@apoorvdarshan); the implementations differ because the table-based dispatch makes each one smaller. Closes pytest-dev#14705 Closes pytest-dev#14716 Closes pytest-dev#11502 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Passing -c/--config-file set the rootdir to the config file's parent directory. A config kept in a subdirectory -- the common `-c config/pytest.ini` layout -- therefore moved the rootdir into that subdirectory, which broke conftest discovery and node ids: conftests collected an empty path prefix, so same-named fixtures from sibling directories shadowed each other and node ids came out as `config::test2` instead of `tests/tests2/test2.py::test2`. The config file's location is evidence about where the project lives, not a decision on its own. Take the common ancestor of the config file's directory and the collected test paths instead, falling back to the invocation directory in place of the test paths when none were given. If the two live in unrelated trees the ancestor degrades to the filesystem root, in which case the config file's directory is kept, as before. This covers cases neither of the two open proposals handled alone: scenario before pytest-dev#14454 pytest-dev#14579 now -c config/pytest.ini (no args) config/ ok config/ ok -c config/pytest.ini tests/ config/ ok ok ok sibling config, invoked from a subdirectory config/ sub/ ok ok unrelated working directory config/ unrel. ok ok pytest-dev#14454 (@EternalRights) found the root cause and drove the design discussion; its `invocation_dir` rule is right for the no-args case but follows the working directory even when that is unrelated to the project. pytest-dev#14579 (@hariharan077) contributed the common-ancestor form, which is most of the answer but leaves the no-args case unfixed. Neither is needed once the ancestor also considers the invocation directory, and no ambiguity warning is required because the ancestor is derived rather than guessed. This is a breaking change: it alters documented behaviour, so doc/en/reference/customize.rst is updated along with it, and the changelog entries are filed as breaking rather than as bugfixes. Closes pytest-dev#13246 Closes pytest-dev#9703 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
psf-chronographer
Bot
added
the
bot:chronographer:provided
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #14807 after review — that PR keeps the refactor and the non-breaking
-cfixes, this one carries the single breaking change.Depends on #14807 (the branch is based on it); the diff against
mainwill shrink to one commit once that lands.Supersedes #14454 and #14579. Closes #13246, closes #9703.
The problem
-cset the rootdir to the config file's parent directory. With the common-c config/pytest.inilayout that moved the rootdir intoconfig/, so conftests collected an empty path prefix: same-named fixtures from sibling directories shadowed each other, and node ids came out asconfig::test2rather thantests/tests2/test2.py::test2.The change
The config file's location is evidence about where the project lives, not a decision on its own. Take the common ancestor of the config file's directory and the collected test paths, using the invocation directory in their place when no paths were given. If the two live in unrelated trees the ancestor degrades to the filesystem root, in which case the config file's directory is kept, as today.
-c config/pytest.ini(no args)config/config/-c config/pytest.ini tests/config/config/sub/config/invocation_dirrule is right for the no-args case but follows the working directory even when that is unrelated to the project. Its ambiguity warning is also written toParser.extra_info, which is only rendered byPytestArgumentParser.error()— so it never appears on a successful run, and its tests pass by asserting onconfig._parser.extra_infodirectly.if dirsand so leaves the no-args case unfixed.No ambiguity warning is needed once the ancestor also considers the invocation directory, because the rootdir is then derived rather than guessed.
This changes documented behaviour —
doc/en/reference/customize.rstsaid "If-cis passed in the command-line, use that as configuration file, and its directory asrootdir" — so the doc is updated here and the changelog entries are filed asbreaking, notbugfix.@bluetech asked on #14807 whether the next release is intended to be major. The
versionchangeddirective in the doc currently says10.0on that assumption — tell me the target and I will adjust it. If the next release is 9.2 instead, this PR should simply wait.Concretely, what breaks: a project that today runs
-c cfg/pytest.iniand relies on the rootdir beingcfg/— node ids,.pytest_cachelocation, and rootdir-relative ini paths all move. In practice such a setup already has broken conftest discovery, which is what #13246 and #9703 report.Verification
Full suite green (4349 passed, 47 skipped, 13 xfailed, 7 xpassed — the xpasses are pre-existing, #11603/#10042).
Verified against the original reproducers by swapping in
main'sfindpaths.pyand running the same commands: onmainthe #13246 repro givesFAILED config::test2 - assert 1 == 2androotdir: .../config; with this commit, 2 passed androotdir: .../— the project root.