Add runtime generation of device interface - #13
Merged
Conversation
bruno-f-cruz
marked this pull request as ready for review
July 26, 2026 03:02
Narrow the Any-typed inputs of format_bulk / to_buffer / to_file to
PayloadBase | ArrayLike, ArrayLike | None, and MessageType | ArrayLike.
Give Device a base REGISTER_MAP ClassVar default ({}) that generated
subclasses override, and tidy the create_device docstring.
bruno-f-cruz
force-pushed
the
feat-runtime-register-binding
branch
from
July 26, 2026 05:08
c1e4787 to
e4cb977
Compare
Add a "Generating a Device from a Schema" example walking through `create_device`, including a win/loss discussion of runtime generation vs. pre-generated device packages. Wire it into the examples index and nav, surface `create_device`/`parse_device_schema`/`ConverterContext` in the device API reference, and add a Quickstart to the root README.
bruno-f-cruz
force-pushed
the
feat-runtime-register-binding
branch
from
July 26, 2026 16:57
de70211 to
933071c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #12.
Summary
Adds a runtime compiler that turns a Harp
device.ymlinto typedRegisterBase/Deviceclasses, mirroring the upstream C# generator (harp-tech/generators, Python target in PR #119) but emitting descriptors at runtime instead of generating static source code. A user with only adevice.ymlgets typed registers and a device, and the C# generator still stays authoritative for shipped device packages.All
device.ymlknowledge converges inharp-device(harp.device._schema).harp-protocolgains only reusable primitives (format_bulk);harp-benchmarksis refactored onto them.Public API
Registers are reached by address through
REGISTER_MAP— they are not attached asDeviceattributes (something to potentially improve in the future?).Custom
interfaceTypes are injected, not stubbed — the runtime analog of the generator'sfrom .converters import <Name>contract:Missing a converter raises
UnknownConverterErrorwhenstrict=True(default);strict=Falsefalls back to native type decoding (e.g: u16). Values are aConverterinstance or a(ConverterContext) -> Converterfactory (the ctx carries the resolved element dtype/size, mask, length, name — so a factory buildsStringConverter(span)/HarpVersionConverter(element)with the right args).exclude_private=Truedrops registers whose DSLvisibilityisprivate.Change surface
harp-protocolRegisterBase.format_bulk(values, *, timestamps=None, message_type=Event, port=255)— the exact vectorized inverse ofparse_bulk(payload encode + header/stride + optional timestamp block + checksum in one pass). Also the typed-corpus generator used by benchmarks and fixtures.GroupMaskdecode now preserves undefined codes as their raw value, matching the C# generator's unchecked enum cast (Interface.cs). Previously an in-range gap code decoded to-1, a code above the largest memberIndexError'd, and the scalar accessor raised. Now — consistently across scalar and batch — a defined code yields the enum member and an undefined code is kept as the raw int (its own category in a dataframe, never NaN). A per-field_lookup_safeflag keeps the bounds-check-free gather for narrow masked sub-fields (the ones in high-rate registers); whole-element masks over sparse enums take a guarded gather.harp-device— new privateharp.device._schemasubpackage:_model.py— pydantic v2 object model ofdevice.yml(hand-maintained; regeneration command against the pinned upstreamregisters.jsonrecorded in the module). There is a TODO here in the future to ensure this doesnt drift by using a json-pydantic generator_emit.py—parse_device_schema(text)andcreate_registers(...), built on a single_INTERFACESconverter table (every built-in interfaceType — the 9 primitives plusstring/bool/HarpVersion._emit_device.py—create_device(...)façade.create_device/parse_device_schema/ConverterContext(+ the converter aliases) are the public exports.harp-benchmarks— corpora are now random per-frame viaformat_bulk(seeded per register, bytes held to 0..127) instead of formatting one frame and byte-repeating it.BenchmarkedRegisteris now(name, register, timestamped); the hand-authoredvaluefixtures are gone.Divergence from the C# generator
AnalogInput0,DIO0), whereas the generator snake_cases members and UPPER_SNAKEs enum members. Deliberate: it avoids reimplementing and keeping in sync Bonsai.Harp'sFirmwareNamingConvention. The yml is the single source of truth for names.converter: PayloadandRawPayloademit identical behaviorTesting
device.ymlreproduces PR #119'sexpected_device.pyregister-for-register;common.ymlmatchesexpected_core.py(structural forDeviceName, whose core metadata the generator enriches to a string).bytes(emitted.format_bulk(r)) == bytes(oracle.format_bulk(r)), andparse_to_dataframeequality after a positional column rename).test_payload.py,test_register_modeling.py).Not in this PR
harp.data#10) —harp.data.create_reader(device.Device)producing per-register DataFrame readers. This PR emits the device; the reader-factory ergonomics on top of it are follow-up work tracked in Add device-driven reader factory forharp.data#10. For now, read withharp.data.parse_to_dataframe(Dev.REGISTER_MAP[address], source).epoch→DatetimeIndexconventions are intentionally omitted; We should consider this last point for future refactoring of the harp-data package