Skip to content

Align the runtime emitter with the generator naming convention - #19

Merged
glopesdev merged 8 commits into
mainfrom
fix-runtime-generator-naming
Aug 12, 2026
Merged

Align the runtime emitter with the generator naming convention#19
glopesdev merged 8 commits into
mainfrom
fix-runtime-generator-naming

Conversation

@bruno-f-cruz

@bruno-f-cruz bruno-f-cruz commented Aug 10, 2026

Copy link
Copy Markdown
Member

Closes the naming divergence #13 introduced deliberately. That PR kept schema identifiers verbatim and recorded the cost in its own description. harp.device._schema now produces the same identifiers a statically generated package does, so code written against either lines up name for name.

The divergence mattered more after #17. Registers are now reached by name off the module, so a schema-built device and a generated one are meant to be interchangeable at the call site, and identifiers were the last place they were not.

What this changes

Payload fields become snake_case and enum members SCREAMING_SNAKE_CASE:

behavior = create_device_module(Path("device.yml").read_bytes())

payload = behavior.StartPulse.payload_class(
    digital_output=PwmPort.PWM1,        # was DigitalOutput=PwmPort.Pwm1
    pulse_width=np.uint16(300),         # was PulseWidth=...
)

parse_to_dataframe(behavior.AnalogData, buf).columns
# ['analog0', 'analog1', 'analog2', 'accelerometer_0', ...]

Type-level names are untouched. Register classes, enum classes and {Name}Payload stay verbatim from the yml, which is what the generator emits too. This is a breaking change for anything reading a runtime-emitted register by field name, including dataframe columns and payload keyword construction.

The hand-written models in harp-benchmarks move to the same convention, so identifiers are consistent across the repository.

Where the convention comes from

New harp.device._schema._naming, a port of FirmwareNamingConvention.Apply in harp-tech/generators. The Python target defines no casing of its own. Python.cs routes every identifier through that one class, GetPythonFieldName being Apply(name).ToLowerInvariant(), and PyDevice.tt only prints names already computed. Porting anything else would have been guessing at the generator rather than matching it.

Two public functions named for what they produce rather than for the C# class they came from, since nothing here concerns firmware:

enum_member_name("DIPort0")   # 'DI_PORT0'
field_name("DutyCycle")       # 'duty_cycle'

The algorithm is not a word-boundary regex. A run of capitals stays one word and a trailing digit never separates, so TestDIPort1 is TEST_DI_PORT1, PortDIO1 is PORT_DIO1, and DIO0 is unchanged. Two details of the C# are reproduced rather than tidied: the regex match index falls on the separator when one is present, and the replacement lookahead reads the pre-substitution string.

What deliberately keeps its YML name

A custom converter symbol derives from the pre-rename key, matching Python.cs capturing converterBaseName before renaming, so member Data still resolves DataConverter rather than dataConverter. ConverterContext.name therefore stays verbatim and the factory contract from #13 is unchanged.

Payload members carry a reserved prefix

Lowercasing field names put them in the same namespace as the payload API, where a schema field would silently replace a method. A field named ToColumns would have overwritten to_columns, and one named RawPayload would have made format emit a short frame with no error anywhere.

Payload members therefore take a payload_ prefix or become private, and a field name is barred from the prefixes rather than from a list of the members that exist today:

payload_dtype        the record dtype
payload_array        the backing record array
payload_from_buffer  bytes   -> payload
payload_as_columns   payload -> columns

This is a breaking change to harp-protocol. It also closes a case no list could have covered. A field renaming to a Python keyword, such as Break becoming break, is now rejected rather than producing an attribute reachable only through getattr.

Payload class sharing follows the generator

A structured register declaring an interfaceType names its payload after that type and registers sharing the type share one class, as both generator targets do. An earlier revision of this PR added a size check on that reuse. That check rejected two identical registers whose payloadSpec spans several elements without a declared length, which is how Rgb0 and Rgb1 in device.behavior are written, and it passed a genuine mismatch whose total width happened to agree. The reuse is now unvalidated here exactly as it is in the generator, and the underlying gap is recorded at harp-tech/generators#122.

Schema text or bytes

parse_device_schema and create_device_module accept str or bytes, so a schema can be read with read_bytes() and the YAML stream declares its own encoding. Reading it with read_text() and no explicit encoding follows the locale instead, which silently mangles a non-ASCII description under cp1252 and fails to parse at all under latin-1.

Testing

Full suite passes at 357 tests, with ruff, pyright, codespell and mkdocs build --strict clean.

  • Convention unit tests. 67 cases in test_naming.py, every pair lifted from the Python generator committed expected output read against its test metadata, so the port is pinned to the C# behavior rather than to a re-derivation of it. Plus fixed-point and degenerate inputs.
  • Reference comparison is now name-exact. _layout() in test_emit.py no longer strips field names, so the existing register-for-register comparison against expected_device.py covers identifiers alongside byte layout. All 15 device registers match, and enum parity is asserted per enum-backed field.
  • Bulk round-trip strengthened. The cross-read no longer renames oracle columns positionally and decode_enums is back on, so emitted and generated classes are compared on column names and decoded enum labels rather than raw codes by position.
  • Payload sharing and rejection. New tests cover an anchored merge sharing one payload class, an offset-sized payload with no declared length, and field names that take the reserved prefix or rename to a keyword.
  • expected_device.py is unchanged.

Reuse a cached payload class whenever a structured register names the
same interfaceType, matching the payload lookup both generator targets
perform. The previous check compared the cached payload against the
element size times the declared length, which is only how a payload is
sized when a length is given; a payloadSpec spanning several elements
without one takes its size from the member offsets instead, so two
identical registers sharing an interfaceType were rejected. The check
also passed a genuine mismatch whose total width happened to agree.

Cover the reuse with the anchor and merge form the published schemas
use, and with an offset-sized payload carrying no declared length.
Payload members take a payload_ prefix or become private, so the public
surface is payload_dtype, payload_array, payload_from_buffer and
payload_as_columns. A field name is barred from those prefixes rather
than from a list of the members that exist today, and is also rejected
when it is a Python keyword or not a valid identifier.

A schema field can no longer shadow a payload member. Keys such as
ToColumns, RawPayload and Dtype now decode to ordinary fields, where
before they replaced the member of that name, and a key such as Break
is rejected rather than producing a field reachable only through
getattr.

A rejected field or an unusable register is reported against the
register as the schema spells it, rather than against the payload class
name the emitter derives from it.
parse_device_schema and create_device_module take str or bytes, so a
schema can be read with read_bytes() and the YAML stream declares its
own encoding. Reading it with read_text() and no explicit encoding
follows the locale instead, which silently mangles a non-ASCII
description under cp1252 and fails to parse at all under latin-1.

The benchmark report and its README name payload_as_columns, matching
the method they measure.
@glopesdev glopesdev changed the title Ensure name parity between static and runtime generators Align the runtime emitter with the generator naming convention Aug 11, 2026

@bruno-f-cruz bruno-f-cruz left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small issue about a public interface/naming, otherwise looks good.

Comment thread src/packages/harp-protocol/src/harp/protocol/_payload.py Outdated
The auto-generated batch sibling is _PayloadBatchType rather than
_batch, so the attribute reads as the class it holds. It stays private,
since the Batch protocol is what callers annotate against.
@glopesdev
glopesdev merged commit 87051fb into main Aug 12, 2026
13 checks passed
@glopesdev
glopesdev deleted the fix-runtime-generator-naming branch August 12, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants