Skip to content

Emit runtime device interface as a module - #17

Merged
glopesdev merged 7 commits into
mainfrom
refactor-register-modules
Aug 10, 2026
Merged

Emit runtime device interface as a module#17
glopesdev merged 7 commits into
mainfrom
refactor-register-modules

Conversation

@glopesdev

Copy link
Copy Markdown
Contributor

This is an alternative to #16, stacked on the same #15 head, so the diff is the single commit that separates the two designs. It gives registers the by-name access #16 adds, using the module they already live in, and keeps REGISTER_MAP as the address view.

What this changes

create_device becomes create_module and returns a module rather than a Device subclass:

behavior = create_module(Path("device.yml").read_text())

behavior.AnalogData            # by name
behavior.REGISTER_MAP[44]      # the same class, by address
behavior.WHO_AM_I              # device identity, from the schema

The schema registers are merged with the common Harp ones at module level, so a schema-built device is reached exactly the way a generated package is. The module is not registered in sys.modules, so the caller binds it, and two schemas may share a device name without clashing.

Device carries no register collection at all. read, write and subscribe already take a register class, and subscribe records the class the caller passed, so nothing inside harp-device ever needed to look a register up by address.

Why a module

#16 requires a generated package to emit each register at module scope, and then a namespace class re-declaring every one of them as AnalogData = AnalogData. The module namespace produced by the first step is already a complete mapping from name to class; the second step builds a second one beside it so that it can be typed. A module is the container Python already has for a collection of types, and every type checker and REPL handles it without help.

What that removes: RegisterMap and CoreRegisters, 203 lines whose job is attribute access, by_name, by_address, iteration, in, len and __dir__; the Device type parameter and its PEP 696 default, threaded through harp-device, harp-data and harp-serial; and the typing-extensions runtime dependency that default requires. Measured under src/, this branch is +205 / -369 against #16.

A module is honestly worse in one place. One built at runtime is invisible to a type checker, so the schema-driven path gets no static types. #16 resolves those names to type[RegisterBase[Any]], so read returns ParsedHarpMessage[Any] on that path in either design and the payload type is unknown both ways. The difference is confined to the statically generated path, where a real module on disk autocompletes and type-checks with no machinery behind it.

What downstream code generators must emit

Nothing new. Register classes at module scope and a REGISTER_MAP spreading the core map, which is what they emit today, plus optionally a Device subclass setting __whoami__.

tests/device/expected_device.py is a checked-in sample of generator output and is unchanged by this branch. harp-tech/generators#119 needs no change, whereas #16 forbids it from emitting REGISTER_MAP and requires the namespace class instead.

Collision rule

A device register now displaces the common register it collides with from both views at once, by address or by name. A schema claiming address 0 leaves its own register at REGISTER_MAP[0] and drops WhoAmI from the module, so a name and the address map can never disagree about what sits at an address. The earlier behavior would keep both names live and pin only the address half.

Dataset reader

DatasetReader takes a module in place of a device class and reads REGISTER_MAP off it, so reader.read still accepts either a register class or an address. create_dataset_reader is unchanged from the outside.

Testing

The full suite passes at 267 tests, with ruff, pyright, codespell and mkdocs build --strict clean. Every assertion in the old test_device_emit.py carries over to test_create_module.py, with new cases for name reachability, __all__, the sys.modules guarantee, and the invariant that the name and address views agree.

create_module replaces create_device and returns a module instead of a
Device subclass. The schema registers, merged with the common Harp
ones, sit at module level keyed by name, beside a REGISTER_MAP keyed by
address and the schema identity as WHO_AM_I, so a schema-built device
is reached the same way a generated package is, with
behavior.AnalogData resolving the same class as
behavior.REGISTER_MAP[44]. The module is not registered in sys.modules
and has to be bound by the caller.

A device register now displaces the common one it collides with from
both views at once, by address or by name, so a name and the address
map can no longer disagree about what sits at an address. Device
carries no REGISTER_MAP of its own, and DatasetReader takes a module in
place of a device class.
@glopesdev
glopesdev requested a review from bruno-f-cruz August 7, 2026 17:04
Comment thread src/packages/harp-device/src/harp/device/_emit_module.py
Comment thread src/packages/harp-device/src/harp/device/_emit_module.py Outdated
Comment thread src/packages/harp-device/src/harp/device/_emit_module.py
create_module now returns a DeviceModule, a ModuleType subclass that
declares REGISTER_MAP, WHO_AM_I and, through __getattr__, the register
names the schema supplies. Register access resolves as
type[RegisterBase[Any]] rather than Any, and a checker that infers
without typeshed resolves the members instead of reporting
behavior.AnalogData as a missing attribute. An absent name raises
AttributeError naming the module and the register.

DatasetReader reaches REGISTER_MAP dynamically, since it also accepts a
generated device package, which is a plain module carrying no such
declaration. The emitter module is renamed _emit_module, after what it
now builds.
pyright now covers docs/examples and tests/conformance.py, a module of
assert_type fixtures pinning the types the documented API resolves to.
The suites stay out, since they pass registers deliberately wrong values
and import private symbols, and tests was previously excluded outright,
which suppressed those paths even when passed explicitly on the command
line.

Checking the examples caught five errors in files nothing had been
checking. A TimestampSeconds handler now takes the uint32 payload the
register parses to rather than float, and OperationControlPayload is
built with every field, using EnableFlag members where it declares them.
@glopesdev
glopesdev requested a review from bruno-f-cruz August 9, 2026 21:10
create_module becomes create_device_module, matching the DeviceModule it
returns and the create_dataset_reader convention in harp-data, and
keeping the meaning at a call site that imported it directly. The
example directory and the test module follow the function name.

The harp-device README now describes a device module by what it holds:
the identity as WHO_AM_I, the register classes at module level, and a
REGISTER_MAP expanding the core one, which is the structure both the
generator and create_device_module produce. A WHO_AM_I of 0 is
documented as an unregistered device whose identity checks are skipped,
and the core module carries none. Subclassing Device is no longer
presented as the way to extend a device, only as what validates WhoAmI
on connect until identity is read from the module.

@bruno-f-cruz bruno-f-cruz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Mostly just wondering if we should abandon the bare "module" name in favor of "*_device_module" everywhere. Otherwise small question regarding a type hint.

Comment thread docs/examples/subscribing_to_events/subscribing_to_events.py
Comment thread src/packages/harp-data/src/harp/data/_dataset.py Outdated
Comment thread src/packages/harp-data/src/harp/data/_dataset.py Outdated
DatasetReader takes device_module rather than module, exposes it as
DatasetReader.device_module, and hints it as DeviceModuleLike, a
protocol matching any module that carries __name__ and REGISTER_MAP.
A generated device package is a plain module, so a nominal hint would
reject it. Matching structurally accepts both it and DeviceModule, and
rejects a module following neither. Registers are read as an attribute
rather than through getattr.

The harp-protocol README documents why register values are numpy
scalars, beside the parse example that first returns one. The value
carries the width its register declares, which a Python int has no
way to represent.

@bruno-f-cruz bruno-f-cruz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two final small notes and ready to merge this and all the underlying PRs.

Comment thread src/packages/harp-device/src/harp/device/_emit_module.py
Comment thread src/packages/harp-device/src/harp/device/_emit_module.py Outdated
A device module now names only the registers its schema declares, so
REGISTER_MAP carries the common ones without the namespace repeating
them. A dataset reader still decodes common registers, and this matches
what a generated package already produces.

DeviceModuleLike also requires WHO_AM_I, so the common register set no
longer satisfies it and cannot be passed where a device module is
expected.
create_device_module accepts str alone, dropping the DeviceModel
overload.

The parameter is renamed from source to text, matching the sibling
parse_device_schema. The word source is what the path-taking convention
uses, so it suggested a path where contents were expected.
@glopesdev
glopesdev requested a review from bruno-f-cruz August 10, 2026 17:56
@glopesdev
glopesdev force-pushed the feat-add-device-reader branch from 72522ae to 94a1c70 Compare August 10, 2026 18:30
Base automatically changed from feat-add-device-reader to main August 10, 2026 18:33
@glopesdev
glopesdev force-pushed the refactor-register-modules branch from 786683b to edeb2f9 Compare August 10, 2026 18:58
@glopesdev
glopesdev merged commit 7892e85 into main Aug 10, 2026
13 checks passed
@glopesdev
glopesdev deleted the refactor-register-modules branch August 10, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants