Skip to content

feat(engine): add transient in-memory graphs - #500

Open
mrkorchun wants to merge 3 commits into
ModernRelay:mainfrom
mrkorchun:agent/in-memory-graph-backend
Open

feat(engine): add transient in-memory graphs#500
mrkorchun wants to merge 3 commits into
ModernRelay:mainfrom
mrkorchun:agent/in-memory-graph-backend

Conversation

@mrkorchun

@mrkorchun mrkorchun commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What changed

  • add Omnigraph::in_memory(schema) for isolated transient embedded graphs
  • reuse Lance native shared-memory object store instead of adding another backend implementation
  • keep the shared-memory URI private and preserve the existing local/S3 storage API
  • add an insert then select integration test

Why

Embedded runtimes need the normal typed graph operations without filesystem hard-link requirements or a durable graph directory. This gives them the same schema, mutation, query, manifest, and branch machinery as durable graphs while keeping lifecycle explicitly process-local.

Validation

  • cargo fmt --all -- --check
  • git diff --check
  • focused lifecycle test added; GitHub CI is the execution gate because the available local Cargo targets were cold and a full Android/Termux rebuild was intentionally not repeated

Known ceiling

Lance retains shared-memory authorities for the process lifetime. Explicit reclamation is deferred until long-lived embedded hosts demonstrate that it is needed.

Greptile Summary

The PR adds isolated, process-local in-memory graphs while preserving the existing graph initialization, manifest, mutation, query, and branch machinery.

  • Adds Omnigraph::in_memory(schema_source) using Lance’s native shared-memory URI scheme.
  • Extends URI and object-path handling for shared-memory datasets.
  • Registers the constructor in the closed public API registry.
  • Documents the transient lifecycle and adds an insert/query integration test.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the public API registry now classifies the constructor, and the owning user documentation describes its transient and non-reopenable lifecycle.

Important Files Changed

Filename Overview
crates/omnigraph/src/db/omnigraph.rs Adds the documented transient constructor and routes it through the established initialization machinery with an isolated ULID-backed authority.
crates/omnigraph/src/storage.rs Preserves shared-memory URIs during joining and normalization while leaving the public local/S3 storage-selection API unchanged.
crates/omnigraph-storage/src/lib.rs Adds shared-memory URI path decoding for the existing in-memory object-store adapter.
crates/omnigraph/src/db/manifest/layout.rs Keeps shared-memory table locations as native URIs instead of converting them to filesystem URLs.
crates/omnigraph/src/db/manifest/metadata.rs Converts shared-memory dataset URIs into object-store-relative metadata paths.
crates/omnigraph/tests/forbidden_apis.rs Correctly classifies in_memory as a bootstrap write surface, resolving the previously reported registry failure.
crates/omnigraph/tests/lifecycle.rs Covers initialization, typed mutation, and query behavior for a transient graph.
docs/user/concepts/storage.md Resolves the previously reported documentation gap by describing discoverability, process-local lifetime, non-reopenability, and supported usage.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Caller[Embedded Rust caller] --> Constructor[Omnigraph::in_memory]
    Constructor --> URI[Private shared-memory URI]
    Constructor --> Init[Shared graph initialization path]
    Init --> Metadata[In-memory schema and recovery metadata]
    Init --> Lance[Lance shared-memory datasets]
    Lance --> Operations[Typed mutation, query, manifest, and branch operations]
    Operations --> Exit[State discarded at process exit]
Loading

Reviews (3): Last reviewed commit: "docs(storage): document transient graphs" | Re-trigger Greptile

Context used:

Comment thread crates/omnigraph/src/db/omnigraph.rs
Comment thread crates/omnigraph/src/db/omnigraph.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant