feat: support oci:// model URIs via llmman serve

[ericcurtin]

What

Adds an oci:// scheme to parameters.uri, so a model published as a CNCF ModelPack artifact can be served directly:

{
  "name": "my-model",
  "implementation": "mlserver_sklearn.SKLearnModel",
  "parameters": {"uri": "oci://ghcr.io/org/model:tag"}
}

Model distribution is increasingly moving to OCI registries -- the same registries, credentials, mirroring and air-gap tooling a deployment already uses for container images.

How

get_model_uri in mlserver/utils.py is the single dispatch point. Resolution runs before the urlparse, so the well-known-filename search and to_absolute_path apply to the extracted directory unchanged (llmman returns an absolute path, which os.path.join already passes through). The blocking pull runs in an executor so the event loop is not stalled.

Every other uri shape reaches exactly the branch it did before: s3://, gs:// and other rclone remotes are still returned untouched for the runtime to handle; file:// and bare relative/absolute paths are unaffected. tests/test_utils.py passes unchanged, which pins that.

Acquisition is delegated to a running llmman serve rather than hand-rolled: llmman already implements the ModelPack media types, registry auth, resumable blob download and a content-addressed store.

New mlserver/llmman.py is the daemon client, stdlib-only (urllib), no new dependency:

• GET /api/version probes reachability and identity -- a server answering without a version field is reported as "not an llmman daemon", worth distinguishing from nothing listening.
• POST /api/pull streams NDJSON so a multi-gigabyte fetch is not silent. An error arrives in-band at HTTP 200, and a stream that ends without success is also a failure -- both are errors, not a completed pull.
• llmman resolve --no-pull reports where the bytes landed; --no-pull keeps the daemon the only thing that touches the network.
• LLMMAN_HOST is honoured with llmman's own parsing, including rewriting a wildcard bind (0.0.0.0, [::]) to loopback.

A pull needs both the daemon reachable and the binary on PATH (or MLSERVER_LLMMAN_BIN); each missing piece has its own actionable error, and neither is required unless an oci:// uri is used.

Testing

New tests/test_llmman.py, running against a real HTTP server on a loopback port rather than mocks, so the NDJSON contract is genuinely exercised.

$ pytest tests/test_llmman.py tests/test_utils.py
51 passed

All executed here. 31 new: /api/version accepted, a non-llmman server rejected, nothing-listening reported actionably; pull success with forwarded byte progress and the exact request body asserted; in-band error at HTTP 200; a stream ending without success; non-OK status; a non-JSON diagnostic tolerated; scheme detection incl. case-insensitivity; that s3://, gs://, file://, a bare filename and an absolute path are not claimed; the resolve contract plus seven malformed-output cases; every LLMMAN_HOST form incl. wildcard-to-loopback; and two get_model_uri integration tests -- an oci:// uri resolved to a local path, and an s3:// uri never reaching the resolver.

tests/test_utils.py (20 tests, incl. the full get_model_uri parametrisation) passes unchanged, so no existing uri behaviour regressed.

• black clean on all three files

Not verified here, flagged rather than implied: no end-to-end model load against a live llmman serve backed by a real registry.

Disclosure: written with AI assistance.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}