Add generic RT/NRT shared resource system (#147) by kaoskorobase · Pull Request #152 · samplecount/methcla

kaoskorobase · 2026-06-01T21:35:51Z

Summary

Adds Methcla_ResourceDef to the plugin C API (include/methcla/plugin.h) — a struct describing a named resource type with uri, instance_size, options_size, mutability hint, and configure/construct/destroy callbacks. Plugins register definitions via methcla_host_register_resource_def.
Adds ResourceId, allocResourceId(), freeResourceId(), resourceNew(), and resourceFree() to the C++ client API (include/methcla/engine.hpp).
Adds OSC commands /resource/new <requestId:i> <resourceId:i> <uri:s> and /resource/free <id:i> with notifications /resource/ready <id:i>, /resource/error <id:i> <msg:s>, and /resource/destroyed <id:i>.
Construction runs on the NRT worker thread; freePending handles the race where /resource/free arrives while construction is still in progress.
Adds AsyncLatch (condvar + timeout) to tests/methcla_tests.hpp as a robust replacement for sleepFor() in async tests.

Deferred

Synth-side kMethcla_ResourcePort integration
AudioBuffer built-in plugin (perform_with_resources host primitive)

Test plan

ResourceTests.UnknownUriReturnsError — unknown URI → /resource/error
ResourceTests.KnownUriConstructsAndNotifiesReady — registered URI → /resource/ready
ResourceTests.FreeLiveResourceDestroysIt — free after ready → /resource/destroyed
ResourceTests.FreeDuringConstructSetsFreePending — free during construction → /resource/destroyed only (no /resource/ready)
Full suite (ctest --preset debug): 19/19 passed

Introduces Methcla_ResourceDef plugin API for defining named resource types, OSC commands /resource/new and /resource/free, and /resource/ready, /resource/error, /resource/destroyed notifications. Supports freePending to handle races between in-flight construction and a concurrent free request. AsyncLatch condvar helper replaces sleepFor() in tests.

kaoskorobase · 2026-06-02T16:40:14Z

+        }
+    };
+
+    inline static std::ostream& operator<<(std::ostream&     out,


Consider implementing this in the Id base class, similar to how it's done in StrongId. Same for NodeId.

kaoskorobase · 2026-06-02T16:41:33Z

+        void resourceNew(ResourceId id, const char* uri)
+        {
+            beginMessage();
+


Consider getting rid of this newline (same in similar Request methods).

kaoskorobase · 2026-06-02T16:42:35Z

+
+            oscPacket()
+                .openMessage("/resource/new", 3)
+                .int32(0) // requestId (unused)


requestId is not needed, remove it from the OSC protocol.

kaoskorobase · 2026-06-02T16:47:02Z

            return m_audioBusIds;
        }

+        ResourceId allocResourceId()


Follow the existing pattern (see nodeIdAllocator).

kaoskorobase · 2026-06-02T16:47:13Z

Follow the existing pattern (see nodeIdAllocator).

kaoskorobase · 2026-06-02T16:50:49Z

                     const char* message);
+
+    //* Register a resource type definition.
+    void (*register_resource_def)(Methcla_Host*              host,


Move this between register_synthdef and register_soundfile_api. Binary compatibility is not a concern at this point.

kaoskorobase · 2026-06-02T16:56:46Z

+
 static inline void
 methcla_host_register_synthdef(Methcla_Host*           host,
                               const Methcla_SynthDef* synthDef)


Rename to def for consistency.

kaoskorobase · 2026-06-02T16:57:10Z

Rename to def for consistency.

kaoskorobase · 2026-06-02T16:57:44Z

@@ -138,6 +139,9 @@ namespace Methcla { namespace Audio {
        //* Register SynthDef.
        void registerSynthDef(const Methcla_SynthDef* synthDef);


Rename to def for consistency.

kaoskorobase · 2026-06-02T17:02:23Z

+            const int32_t resourceId = args.int32();
+            if (resourceId < 0 ||
+                static_cast<size_t>(resourceId) >= m_resources.size())
+                return;


Avoid early returns.

kaoskorobase · 2026-06-02T17:07:00Z

+            {
+                entry.freePending = true;
+            }
+            else if (entry.state == ResourceEntry::State::Live &&


I think there's a state transition missing here, according to #147:

refcount>0 → free_pending=true, └────────────────────────────────────── Free │ drain via release path

kaoskorobase · 2026-06-02T17:08:33Z

+}
+
+namespace {
+    struct ResourceDestroyCommand


Prefer a class here, with members private where possible. Prefix members properly.

kaoskorobase · 2026-06-02T17:10:10Z

        }
+        else if (msg == "/resource/new")
+        {
+            class ResourceConstructCommand


I think it's more readable in the long run to avoid inline class definitions. Pull this and others out in an anonymous namespace.

kaoskorobase · 2026-06-02T17:19:34Z

+void EnvironmentImpl::scheduleResourceDestroy(int32_t resourceId)
+{
+    auto& entry = m_resources[resourceId];
+    sendToWorker(


Use the template version.

kaoskorobase · 2026-06-02T17:20:45Z

+            entry.def = nullptr;
+            entry.data = nullptr;
+            self->impl
+                ->sendToWorker<EnvironmentImpl::ResourceDestroyedNotification>(


I think an extra memory allocation can be avoided if this class also functions as the notification.

kaoskorobase · 2026-06-02T17:22:27Z

        std::list<const Methcla_SoundFileAPI*> m_soundFileAPIs;

+        using ResourceDefMap =
+            std::unordered_map<std::string, const Methcla_ResourceDef*>;


We can't use std::string here because it allocates. See definition of SynthDefMap.

kaoskorobase · 2026-06-02T17:24:16Z

+        {
+            auto* self = static_cast<ResourceDestroyCommand*>(d);
+            auto& entry = self->impl->m_resources[self->resourceId];
+            entry.state = EnvironmentImpl::ResourceEntry::State::Free;


Better to fully initialize by assigning a default constructed entry.

kaoskorobase · 2026-06-02T17:24:30Z

+                const Methcla_ResourceDef* def = it->second;
+                auto&                      entry = m_resources[resourceId];
+                entry.state = ResourceEntry::State::Constructing;
+                entry.freePending = false;


Better to fully initialize by assigning a default constructed entry.

kaoskorobase · 2026-06-02T17:26:10Z

            }
        }

+        class ResourceErrorNotification : public Notification


This class is probably not needed in the header and can be moved to the source (encapsulation).

kaoskorobase · 2026-06-02T17:32:48Z

+            sendToWorker<ResourceReadyNotification>(resourceId);
+        }
+
+        void registerResourceDef(const Methcla_ResourceDef* def);


Add context comment (NRT)

kaoskorobase · 2026-06-02T17:33:08Z

+        // notification.
+        void scheduleResourceDestroy(int32_t resourceId);
+
+        class ResourceDestroyedNotification : public Notification


This could be merged with the destroy command to avoid an allocation (see earlier comment). In any case, this is probably not needed in the header and can be moved to the source (encapsulation).

kaoskorobase · 2026-06-02T17:33:49Z

    size_t realtime_memory_size = 1024 * 1024;
    size_t max_num_nodes = 1024;
    size_t max_num_audio_buses = 1024;
+    size_t max_num_resources = 256;


These constants are duplicated in various places. I'm wondering whether there's a clean way to avoid the duplication.

kaoskorobase · 2026-06-02T17:35:14Z

+}
+
+// ---------------------------------------------------------------------------
+// Slice 1: Unknown URI → /resource/error


Remove "Slice" comments. Add comments to explain tests where necessary.

kaoskorobase · 2026-06-02T17:38:05Z

+        return false;
+    });
+
+    // Send /resource/new and /resource/free back-to-back without waiting.


Could be sent in a single bundle. This guarantees they're executed in the same process callback.

- Add kMethcla_UnsupportedResourceTypeError; replace notifyResourceError with throwErrorWith so unknown-URI errors flow through the standard catch/log path instead of a bespoke /resource/error notification - Report out-of-range resource id in /resource/free as ArgumentError - Move ResourceReadyNotification, NodeTreeStatisticsCommand, and RTMemoryStatisticsCommand out of method bodies into the anonymous namespace; rename Command* → *Command for consistent naming - Consolidate operator bool() and operator<< into the IdBase template - Add resourceIdAllocator() to the Engine interface; rename audioBusId() → audioBusIdAllocator() for consistency - Fix Methcla_Host initializer order to match struct field order after register_resource_def was repositioned

kaoskorobase force-pushed the resource-system branch from e9813b8 to c05a501 Compare June 1, 2026 21:39