Skip to content

Add schema-free single-register loader - #9

Merged
glopesdev merged 1 commit into
mainfrom
feat-generic-file-reader
Aug 6, 2026
Merged

Add schema-free single-register loader#9
glopesdev merged 1 commit into
mainfrom
feat-generic-file-reader

Conversation

@bruno-f-cruz

Copy link
Copy Markdown
Member

Summary

Introduces harp.data.read(source), the pyharp counterpart of harp-python's harp.io.read: read one register's binary .bin data into a pandas DataFrame, inferring the layout from the first frame. It is implemented on top of the unified harp.protocol register API instead of a hand-rolled byte parser.

Background — how harp-python loaded data

harp-python offered two layers:

  • Low-levelharp.io.read(file, address=, dtype=, length=, epoch=, keep_type=): read a single register file, infer stride / timestamp / row count from the first message, and decode to the native numpy dtype. It hand-coded the Harp framing math (_dtypefrompayloadtype, stride/offset arithmetic, timestamp handling).
  • High-levelharp.reader.create_reader(device) on top of harp.schema.read_schema + harp.model: parse a device.yml, generate per-register readers at runtime, and decode masks / enums / payloadSpec members from the schema. Files were resolved by the {device}_{address}.bin convention.

What this PR adds

  • harp.data.read(source, *, timestamp=True, message_type=False) — inspects the first frame's header (payload-type byte, length byte, timestamp flag), builds the matching native register from harp.protocol (RegisterU16, RegisterS16Array(addr, length=n), …), and delegates to parse_to_dataframeRegisterBase.parse_bulk. Decodes to native numpy types only (no enum / bit-mask decoding), matching io.read's behavior. source may be a path, bytes, or an open binary file.
  • harp.protocol now exports decode_payload_type, encode_payload_type, and PayloadTypeInfo — the canonical payload-type-byte decoder that read relies on to infer the element type, size, and timestamp flag.

@glopesdev
glopesdev merged commit d708cfd into main Aug 6, 2026
13 checks passed
@glopesdev
glopesdev deleted the feat-generic-file-reader branch August 6, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants