Add runtime generation of device interface

README.md

@@ -50,6 +50,45 @@ pip install harp-data
 
 `harp-benchmarks` (under `src/packages/`) is internal-only and is never published to PyPI.
 
+## Quickstart
+
+Have only a device's `device.yml`? `create_device` compiles it into a typed
+`Device` at runtime — no code-generation step — giving you the device's registers
+(keyed by address) and its identity:
+
+```python
+from pathlib import Path
+from harp.device import create_device
+
+Behavior = create_device(Path("device.yml").read_text())
+Behavior.__whoami__            # device identity from the schema
+AnalogData = Behavior.REGISTER_MAP[44]   # registers are reached by address
+```
+
+The generated device works like any other. **Talk to hardware** over a serial
+transport — `read`/`write` take a register class:
+
+```python
+from harp.serial import open_serial_device
+
+# Use "COMx" on Windows, "/dev/ttyUSBx" on Linux.
+with open_serial_device(Behavior, port="/dev/ttyUSB0") as device:
+    print(device.read(AnalogData).parsed)
+```
+
+...or use the same register classes to **decode recorded data** into a pandas
+DataFrame:
+
+```python
+from harp.data import parse_to_dataframe
+
+df = parse_to_dataframe(AnalogData, "Behavior_44.bin")
+```
+
+See the [Examples](https://harp-tech.org/pyharp/examples/) for full walkthroughs,
+including reading device info, subscribing to events, and working with custom
+interface-type converters.
+
 ## Contributing
 
 harp is a [uv workspace](https://docs.astral.sh/uv/concepts/workspaces/): every package under

docs/api/device.md

@@ -3,6 +3,9 @@
 ---
 
 ::: harp.device.Device
+::: harp.device.create_device
+::: harp.device.parse_device_schema
+::: harp.device.ConverterContext
 ::: harp.device.HarpFramer
 ::: harp.device.ITransport
 ::: harp.device.TransportError

docs/examples/create_device/create_device.md

@@ -0,0 +1,53 @@
+# Generating a Device from a Schema
+
+This example demonstrates how to turn a Harp `device.yml` into a typed
+[`Device`](../../api/device.md) at runtime with `create_device`, without a
+code-generation step. This is the quickest way to get started when you have only a
+device's schema and no pre-generated package for it.
+
+The compiled device exposes its registers through `REGISTER_MAP` (keyed by address)
+and carries the device's `__whoami__` identity. From there it works exactly like a
+pre-generated device class — drive it over a transport to talk to hardware, or use
+its register classes to decode recorded data.
+
+## When to use runtime generation
+
+`create_device` trades statically generated device packages for schema-driven
+convenience. It's worth understanding what that buys you and what it costs.
+
+**You gain:**
+
+- **No build step.** A `device.yml` — even one you just pulled off a device —
+  becomes a working device in a single call. There's nothing to generate, install,
+  or keep in sync with the schema.
+- **Coverage for any device.** You don't need a published package for the device;
+  unreleased, custom, or one-off schemas work immediately.
+- **The schema stays the single source of truth.** Register, field, and enum names
+  come straight from the `device.yml`, verbatim.
+
+**You give up:**
+
+- **Named, typed access.** Registers are reached by address (`REGISTER_MAP[44]`),
+  not as importable, autocompleting classes (`from harp_behavior import AnalogData`).
+  You lose editor discovery and static type checking of register names.
+- **Generator naming conventions.** Identifiers are kept verbatim from the yml
+  (`AnalogInput0`, `DIO0`) rather than the C# generator's snake_case fields and
+  `UPPER_SNAKE` enum members, so code written against a generated package won't line
+  up name-for-name.
+- **Turn-key custom types.** A custom `interfaceType` must be injected yourself via
+  `converters=` (see below), whereas a generated package ships its own converters.
+
+For shipped, widely-used devices a pre-generated package from the
+[Harp C# generator](https://github.com/harp-tech/generators) remains the
+authoritative choice — better editor support and static typing. Reach for
+`create_device` when you want to go from a schema to working code with no
+generation step.
+
+!!! warning
+    Don't forget to change the `SERIAL_PORT` to the one that corresponds to your device! The `SERIAL_PORT` must be denoted as `/dev/ttyUSBx` in Linux and `COMx` in Windows, where `x` is the number of the serial port.
+
+<!--codeinclude-->
+```python
+[](./create_device.py)
+```
+<!--/codeinclude-->

docs/examples/create_device/create_device.py

@@ -0,0 +1,38 @@
+from pathlib import Path
+
+from harp.data import parse_to_dataframe
+from harp.device import create_device
+from harp.serial import open_serial_device
+
+SERIAL_PORT = "/dev/ttyUSB0"  # or "COMx" in Windows ("x" is the number of the serial port)
+
+# `create_device` compiles a Harp `device.yml` into a typed `Device` subclass at
+# runtime — no code-generation step. This is the quickest way to work with a device
+# when you don't have a pre-generated package for it: point it at the schema and you
+# get the device's registers (keyed by address) plus its identity.
+Behavior = create_device(Path("device.yml").read_text())
+
+print("WhoAmI:", Behavior.__whoami__)  # device identity, taken from the schema
+AnalogData = Behavior.REGISTER_MAP[44]  # registers are reached by address
+
+# The generated device behaves like any other `Device` class. Talk to hardware over
+# a transport — `read`/`write` take a register class:
+with open_serial_device(Behavior, port=SERIAL_PORT) as device:
+    print("AnalogData:", device.read(AnalogData).parsed)
+
+# ...or use the same register classes to decode a recorded binary dump into a
+# pandas DataFrame (see the "Reading Data into a DataFrame" example for more):
+df = parse_to_dataframe(AnalogData, "Behavior_44.bin")
+print(df.head())
+
+
+# --- Custom interface types --------------------------------------------------
+# A register with a custom `interfaceType` needs a converter so its field decodes
+# to the right Python type. Pass it via `converters=`, keyed by "<Name>Converter":
+#
+#   Behavior = create_device(yml_text, converters={"DataConverter": DataConverter()})
+#
+# An unresolved custom type raises `UnknownConverterError`; pass `strict=False` to
+# decode it natively instead. `exclude_private=True` (the default) drops registers
+# marked `private` in the schema. If you only want the parsed schema model rather
+# than a device, `parse_device_schema(yml_text)` returns that directly.

docs/examples/index.md

@@ -2,8 +2,16 @@
 
 This section contains some examples to help you get started with `harp`.
 
-Here's the complete list of available examples:
+Working from a device schema:
+
+- [Generating a Device from a Schema](./create_device/create_device.md) - compile a `device.yml` into a typed device at runtime with `create_device`.
+
+Talking to a device:
 
 - [Getting Device Info](./get_info/get_info.md) - connect to a Harp device and read its information.
 - [Read and Write from Registers](./read_and_write_from_registers/read_and_write_from_registers.md) - connect to a Harp device and read and write its registers.
+- [Subscribing to Events](./subscribing_to_events/subscribing_to_events.md) - react to messages pushed by the device without polling.
+
+Reading recorded data:
+
 - [Reading Data into a DataFrame](./read_data_to_dataframe/read_data_to_dataframe.md) - load a register's binary data file into a pandas DataFrame with `harp.data`.

mkdocs.yml

@@ -73,10 +73,11 @@ nav:
   - Home: index.md
   - Examples:
     - examples/index.md
+    - Generating a Device from a Schema: examples/create_device/create_device.md
     - Getting Device Info: examples/get_info/get_info.md
     - Read and Write from Registers: examples/read_and_write_from_registers/read_and_write_from_registers.md
-    - Reading Data into a DataFrame: examples/read_data_to_dataframe/read_data_to_dataframe.md
     - Subscribing to Events: examples/subscribing_to_events/subscribing_to_events.md
+    - Reading Data into a DataFrame: examples/read_data_to_dataframe/read_data_to_dataframe.md
   - API:
     - Protocol: api/protocol.md
     - Serial: api/serial.md

src/packages/harp-benchmarks/src/harp/benchmarks/_registers.py

@@ -1,63 +1,46 @@
 """Shared benchmark fixtures: every register from ``harp.benchmarks.register_models``
-paired with a representative sample value.
+paired with whether its frames carry a timestamp.
 
 Both ``generate.py`` (writes the .bin corpora) and ``benchmark.py`` (times parsing)
-import :data:`BENCHMARK_REGISTERS` from here so the two stay in lock-step: the value
-used to *format* each frame is the same one whose parsed shape we benchmark.
-
-The sample values mirror ``register_models.main()``'s round-trip fixtures — they
-exercise the full spread of payload shapes the Harp protocol allows (trivial
-scalars, struct payloads with byte gaps, masked sub-fields, custom converters,
-enum/flag single-member unwrap).
+import :data:`BENCHMARK_REGISTERS` from here so the two stay in lock-step. Payloads
+are synthesized as random bytes per frame at generation time (see ``generate.py``),
+so no sample values live here — only the register class and its frame shape.
 
 All generated artifacts live under ``./benchmark`` in the current working directory.
 """
 
 from pathlib import Path
 from typing import Any, NamedTuple
 
-import numpy as np
-
 from harp.benchmarks.register_models import (
     AnalogData,
-    AnalogDataPayload,
     BitmaskSplitter,
-    BitmaskSplitterPayload,
     ComplexConfiguration,
-    ComplexConfigurationPayload,
     Counter0,
     CustomMemberConverter,
-    CustomMemberConverterPayload,
     CustomPayload,
     CustomRawPayload,
     DigitalInputs,
     EncoderMode,
-    EncoderModeMask,
-    PortDigitalIOS,
     PortDIOSet,
     PulseDO0,
     PulseDOPort0,
-    PwmPort,
     StartPulse,
-    StartPulsePayload,
     StartPulseTrain,
-    StartPulseTrainPayload,
     Version,
-    VersionPayload,
 )
-from harp.protocol import HarpVersion, RegisterBase
+from harp.protocol import RegisterBase
 
 ARTIFACTS_DIR = Path("benchmark").resolve()
 DATA_DIR = ARTIFACTS_DIR / "data"
 REPORT_PATH = ARTIFACTS_DIR / "report.md"
 
 
 class BenchmarkedRegister(NamedTuple):
-    """A register under benchmark together with a value that ``format()`` accepts."""
+    """A register under benchmark, with whether its corpus frames are timestamped."""
 
     name: str
     register: type[RegisterBase[Any]]
-    value: Any
     timestamped: bool = True
 
     @property
@@ -70,77 +53,28 @@ def filename(self) -> str:
 
 
 def _base_registers() -> list[BenchmarkedRegister]:
-    """One (timestamped) fixture per register — :func:`_build` derives the untimestamped twin."""
+    """One (timestamped) fixture per register — :func:`_build` derives the untimestamped twin.
+
+    The set spans the full spread of payload shapes the Harp protocol allows: trivial
+    scalars, struct payloads with byte gaps, masked sub-fields, custom converters, and
+    enum/flag single-member unwrap.
+    """
     return [
-        BenchmarkedRegister("DigitalInputs", DigitalInputs, np.uint8(0b1010)),
-        BenchmarkedRegister(
-            "AnalogData",
-            AnalogData,
-            AnalogDataPayload(
-                Analog0=np.float32(1.0),
-                Analog1=np.float32(2.0),
-                Analog2=np.float32(3.0),
-                Accelerometer=np.array([4, 5, 6], dtype=np.float32),
-            ),
-        ),
-        BenchmarkedRegister(
-            "ComplexConfiguration",
-            ComplexConfiguration,
-            ComplexConfigurationPayload(
-                PwmPort=PwmPort.Pwm2,
-                DutyCycle=np.float32(0.5),
-                Frequency=np.float32(1000.0),
-                EventsEnabled=True,
-                Delta=np.uint32(42),
-            ),
-        ),
-        BenchmarkedRegister(
-            "Version",
-            Version,
-            VersionPayload(
-                ProtocolVersion=HarpVersion(2, 0, 0),
-                FirmwareVersion=HarpVersion(1, 2, 3),
-                HardwareVersion=HarpVersion(1, 0, 0),
-                CoreId="abc",
-                InterfaceHash=np.arange(20, dtype=np.uint8),
-            ),
-        ),
-        BenchmarkedRegister("CustomPayload", CustomPayload, HarpVersion(3, 1, 4)),
-        BenchmarkedRegister("CustomRawPayload", CustomRawPayload, HarpVersion(0, 0, 1)),
-        BenchmarkedRegister(
-            "CustomMemberConverter",
-            CustomMemberConverter,
-            CustomMemberConverterPayload(Header=np.uint8(7), Data=-1234),
-        ),
-        BenchmarkedRegister(
-            "BitmaskSplitter",
-            BitmaskSplitter,
-            BitmaskSplitterPayload(Low=np.int32(0xA), High=np.int32(0x5)),
-        ),
-        BenchmarkedRegister("Counter0", Counter0, np.int32(-100000)),
-        BenchmarkedRegister(
-            "PortDIOSet",
-            PortDIOSet,
-            PortDigitalIOS.DIO0 | PortDigitalIOS.DIO3,
-        ),
-        BenchmarkedRegister("PulseDOPort0", PulseDOPort0, np.uint16(5)),
-        BenchmarkedRegister("PulseDO0", PulseDO0, np.uint16(9)),
-        BenchmarkedRegister(
-            "StartPulse",
-            StartPulse,
-            StartPulsePayload(DigitalOutput=PwmPort.Pwm1, PulseWidth=np.uint16(300)),
-        ),
-        BenchmarkedRegister(
-            "StartPulseTrain",
-            StartPulseTrain,
-            StartPulseTrainPayload(
-                DigitalOutput=PwmPort.Pwm1,
-                PulseWidth=np.uint16(300),
-                Frequency=np.uint8(200),
-                PulseCount=np.uint8(50),
-            ),
-        ),
-        BenchmarkedRegister("EncoderMode", EncoderMode, EncoderModeMask.Displacement),
+        BenchmarkedRegister("DigitalInputs", DigitalInputs),
+        BenchmarkedRegister("AnalogData", AnalogData),
+        BenchmarkedRegister("ComplexConfiguration", ComplexConfiguration),
+        BenchmarkedRegister("Version", Version),
+        BenchmarkedRegister("CustomPayload", CustomPayload),
+        BenchmarkedRegister("CustomRawPayload", CustomRawPayload),
+        BenchmarkedRegister("CustomMemberConverter", CustomMemberConverter),
+        BenchmarkedRegister("BitmaskSplitter", BitmaskSplitter),
+        BenchmarkedRegister("Counter0", Counter0),
+        BenchmarkedRegister("PortDIOSet", PortDIOSet),
+        BenchmarkedRegister("PulseDOPort0", PulseDOPort0),
+        BenchmarkedRegister("PulseDO0", PulseDO0),
+        BenchmarkedRegister("StartPulse", StartPulse),
+        BenchmarkedRegister("StartPulseTrain", StartPulseTrain),
+        BenchmarkedRegister("EncoderMode", EncoderMode),
     ]