several things - #348
Merged
Merged
Conversation
Rewrites flopy4/mf6/utils/codegen/filters.py and make.py to source from modflow_devtools.dfns (pydantic, schema 2.0.0.dev3) instead of the legacy flat-TypedDict modflow_devtools.dfn schema, and flips cli.py's sync path to dev3. Bumps the modflow-devtools pin (pixi.lock) to pick up the pk/fk backfill and other fixes this migration depends on. Also merges the originally-separate Phase 0.6b (reflect DFN structure literally) and Phase 0.7 (real union field type) work into this pass where the runtime already supports it: OC's per-rtype table is now read from the schema's real rtype.valid instead of a hardcoded table, and LAK/LKE/LKT/SFR-style keystring period settings are detected directly from the schema's Union arms instead of a hand-maintained override list. A faithful typed-union representation (validated separately, not wired in) is deferred to a follow-up phase once the Row migration (Phase 0.6) lands, since structure.py/unstructure.py only support the flat Schema/ Column/role vocabulary today. Regenerated all existing packages against the new pipeline and fixed what the resulting diff surfaced: dropped `numeric_index`/`optional` migration gaps in devtools (logged upstream, with local overrides as a stopgap), a few flopy4-side misclassifications (string pk/fk shouldn't get numeric index conversion, bare period scalars need the same fill-forward wrapping as period lists), and one duplicate-field bug from a now-stale override. See dev3-migration-gaps.md for the full rundown. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the np.recarray + Schema/Column-driven ingress/egress machinery with a Row mixin (flopy4/mf6/row.py): generated Row/record classes now carry their own field()-level metadata (pk/fk/cellid/tagged/time_series/ prefix/inout) and are the single source of truth for (un)structuring list input, matching the pattern record.py's Record mixin already used for compound records. package.py, converter/ingress/structure.py, and converter/egress/unstructure.py are rewritten to introspect Row classes directly via attrs.fields() instead of a separate Schema/Column lookup; flopy4/mf6/schema.py and its standalone unit tests are removed outright since nothing else references them. Also fixes two real parsing bugs surfaced along the way: row_class() codegen was placing the synthetic `aux` field ahead of a package's own optional non-aux columns (wrong relative to the real DFN token order, e.g. EVT's pxdp/petm/petm0), and Row.from_row() greedily consumed tokens for optional columns without knowing whether they were actually present in a given row, misaligning subsequent fields; from_row() now infers how many trailing optional columns are present from the remaining token budget, the same way naux/ncelldim are already inferred. Regenerates all 64 flopy4.mf6 component modules from the fixed codegen and ports the hand-written tdis.py/disv.py (gwf/gwt/gwe/prt) files off Schema/Column onto Row. Full suite is green except two pre-existing, unrelated integration-test failures traced to xattree mangling an explicit multi_package `name=` (e.g. "LAK-1" -> "LAK-10"), reproduced in isolation with no Row/Column involvement. Also commits the working planning docs this migration is tracked against (namefile-load-plan.md, sync-plan.md, protocols-plan.md, list-design.md, dev3-migration-gaps.md). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Simulation.load("mfsim.nam")/Gwf.load("model.nam") now recursively
resolve real MF6 namefiles instead of silently dropping packages/models/
exchanges/solutiongroup rows. Adds an FTYPES registry (component.py) and
_resolve_bindings() (converter/ingress/structure.py), which finds a
component's child-Component fields via xattree.get_xatspec().children
(mirroring unstructure.py's egress-side _make_binding_blocks), resolves
each namefile row's ftype token to a target class, and recursively loads
it -- merged into structure_component()'s existing single cls(**kwargs)
call rather than a separate post-construction attach step. Dimension
propagation (dis loaded before npf/ic/sto/buy in the same block, dims
threaded into their Package.load(dims=...) calls) is a small local
ordering rule inside _resolve_bindings, not a dimensions.py change.
G/A-variant packages (Chdg, Rcha, etc.) share their base's namefile
ftype -- confirmed against real MF6 source (gwf.f90's package-type
dispatch has no separate case for them), not legacy flopy's
dfn_file_name/_package_type, which describe DFN/class identity rather
than the namefile-level contract. Since this means Union[Chd, Chdg]-style
fields can't be disambiguated by token, _resolve_bindings peeks the
referenced file's OPTIONS block for the READASARRAYS/READARRAYGRID
marker instead, the same signal MF6 itself uses.
Also fixes a dead code path: Component.load()/Context.load() were
routed through converter.structure(), a bare cattrs.Converter with no
structure hook registered for the abstract Component base (never
worked; only mocked-loader tests exercised it). _load_mf6 now uses the
same structure_component() pipeline Package.load() already used
successfully, and the manual child-loading loops are dropped in favor
of children arriving already resolved from construction.
Known limitation: a binding row's pname isn't threaded through to the
loaded child's name (loaded children get xattree's default auto-
assigned name); Solution.models/Exchange.exgmnamea/exgmnameb are,
since those are real semantic data recoverable only from the row.
Renames namefile-load-plan.md to mf6-object-model-plan.md (scope grew
beyond namefile loading into the DFN-schema-migration prerequisite work
that turned out to gate it) and updates its Phase 0.6/0.6a/0.6b/0.7
status -- those had already landed but the doc, added to git in the
same commit that finished them, was never updated to say so. Marks
Phase 1 done with the above. Updates CLAUDE.md's reference to the
renamed file and its own description of current status.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jb3D6Df2Nh84MaJsmcvH3r
CLAUDE.md and the root-level *-plan.md/*-gaps.md/list-design.md planning docs are local working notes, not meant to be version-controlled -- git rm --cached (not git rm), files stay in the working tree, and .gitignore keeps them from being re-added by accident. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jb3D6Df2Nh84MaJsmcvH3r
Follow-up to c8ca43c -- the .gitignore edit didn't make it into that commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jb3D6Df2Nh84MaJsmcvH3r
Component.load()/Context.load()/Package.load()/structure_component() all gain an optional name= param, threaded from a binding row's pname by _resolve_bindings. This closes the "known limitation" from the previous commit for dict-kind fields (Simulation.models/exchanges/solutions): xattree reconciles a dict-kind child's .name to the key it's attached under, so using the row's pname as that key (instead of fname, which turned out not to be purely cosmetic as assumed) makes a custom pname round-trip correctly -- confirmed by test. It does not help list-kind (chd, wel, ...) or only-kind (dis, ic, npf, ...) package fields: xattree reconciles those to a field-derived name regardless of what's passed to the constructor, confirmed true even at write time (Chd(name="custom") already becomes "chd0" on attachment, before any of this code runs). Not a load-side gap this closes without touching xattree's own child-attachment convention, which is out of scope here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jb3D6Df2Nh84MaJsmcvH3r
…torage binding.py maps the Component object model to/from MF6 wire-format tuples, which is what converter/ (structure.py/unstructure.py) already does on the ingress/egress sides of this same job -- codec/ stays a pure syntax layer with no Component knowledge. COMPONENTS and FTYPES each stored two entries per class (a plain key and a model-qualified key), with inconsistent tie-break behavior on collision (last-write-wins vs setdefault's first-write-wins). Both registries now store a single fully-qualified key per class, with lookup_component()/lookup_ftype() resolving a qualified name, an unprefixed name, or a best-effort bare name (returning None rather than an arbitrary class when more than one model shares it).
Codegen already computed the canonical DFN name (e.g. "gwf-ic") per component but discarded it after building the class name/output path. Emit it as a ClassVar on every generated class instead, and have FNAMES register a class only when it declares its own dfn_name -- this replaces the module-path/class-name reconstruction in __attrs_init_subclass__ and naturally excludes abstract bases (Package, Context, Model, Exchange, Solution, DisBase) without the "ABC in cls.__bases__" check that missed DisBase. The handful of hand-written classes (Simulation, Tdis, Gwf/Gwt/Gwe/Prt, per-model Dis/Disv, the GWF exchanges) declare dfn_name by hand using their real DFN identity. Backfilled dfn_name onto the 55 already- generated modules from the current registry, catching one real mismatch along the way: Ims/Ems were keyed as bare "ims"/"ems" under the old module-path scheme, but their DFN names are "sln-ims"/"sln-ems". get_ftypes() now qualifies a ftype token with a model prefix only when the bare token actually collides across classes, rather than always qualifying based on a class's own dfn_name -- naively doing the latter double-prefixed singleton tokens like Gwf's "gwf6" into "gwf-gwf6".
Every case in this file exercised infrastructure that no longer exists: Row.__iter__ (Row classes don't implement it anymore), Package's old np.recarray coercion (stress_period_data/packagedata are now list[Row]), and dtype-based recarray column access (spd["head"][i]). Equivalent coverage of the current list[Row] API (all input styles, multi-row/ multi-period, aux, boundnames, static block rows) already lives in test_mf6_codec.py, test_converter_structure.py, test_dataframe_api.py, and test_mf6_component.py, updated in the prior commit. Full suite now runs clean with no exclusions: 629 passed, 2 pre-existing/unrelated failures. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…E tests Positional tuples bind by Row field declaration order (aux before boundname); the two new tests had them swapped, writing the boundname string into the aux slot and the aux value into the boundname slot. That malformed PACKAGEDATA block was crashing MF6 with SIGSEGV.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
np.recarray+Schema/Columnmachinery with aRowmixinbinding.pyintoconverter/dfn_nameattribute to components