Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/python-merge-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs:
cosmosChanged: ${{ steps.filter.outputs.cosmos }}
duckdbChanged: ${{ steps.filter.outputs.duckdb }}
qdrantChanged: ${{ steps.filter.outputs.qdrant }}
milvusChanged: ${{ steps.filter.outputs.milvus }}
mongodbChanged: ${{ steps.filter.outputs.mongodb }}
redisChanged: ${{ steps.filter.outputs.redis }}
githubCopilotChanged: ${{ steps.filter.outputs.github_copilot }}
Expand Down Expand Up @@ -98,6 +99,9 @@ jobs:
- 'python/packages/qdrant/**'
- '.github/workflows/python-merge-tests.yml'
- '.github/workflows/python-integration-tests.yml'
milvus:
- 'python/packages/milvus/**'
- '.github/workflows/python-merge-tests.yml'
mongodb:
- 'python/packages/mongodb/**'
- '.github/workflows/python-merge-tests.yml'
Expand Down Expand Up @@ -144,6 +148,27 @@ jobs:
- name: Test DuckDB package
run: uv run poe test -P duckdb

python-tests-milvus:
name: Python Tests - Milvus
needs: paths-filter
if: >
github.event_name == 'pull_request' &&
(needs.paths-filter.outputs.milvusChanged == 'true' ||
needs.paths-filter.outputs.coreChanged == 'true')
runs-on: ubuntu-latest
defaults:
run:
working-directory: python
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Set up python and install the project
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
- name: Test Milvus package
run: uv run poe test -P milvus

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job runs only poe test, which excludes the newly added live integration tests. The workflow header requires parallel jobs to stay in sync with python-integration-tests.yml, but that workflow has no Milvus service/Lite job and Milvus is absent from its report/gate dependencies. Please schedule poe test-integration against a disposable Milvus server or Milvus Lite in the integration workflow and wire it into the aggregate report/check.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking into this, I hit two concrete constraints and want your steer before wiring it:

  1. Milvus Lite can't be driven via MILVUS_URI. pymilvus reads a global MILVUS_URI at import and validates it as http(s)://..., so pointing it at a local .db crashes import pymilvus. (I verified AsyncMilvusClient itself works fine against Lite when the path is passed as the uri= argument, not the env var — I've fixed the docs accordingly.)
  2. No single-container Milvus service. Milvus standalone needs etcd+MinIO (or embedded etcd via milvus run standalone), and GitHub services: can't override the image command, so it can't be a service container like qdrant/redis/postgres — it needs a docker run step.

Proposed: add python-tests-milvus to python-integration-tests.yml that docker runs a disposable Milvus standalone (embedded etcd) with MILVUS_URI=http://localhost:19530, waits on /healthz, runs poe test-integration, and wire it into the report + python-integration-tests-check gate. Happy to push that — just confirming you'd prefer the standalone-server route over Milvus Lite (which the reserved env var rules out for the MILVUS_URI-gated tests).


python-tests-qdrant:
name: Python Integration Tests - Qdrant
needs: paths-filter
Expand Down Expand Up @@ -980,6 +1005,7 @@ jobs:
needs:
[
python-tests-duckdb,
python-tests-milvus,
python-tests-unit,
python-tests-openai,
python-tests-azure-openai,
Expand Down
1 change: 1 addition & 0 deletions python/PACKAGE_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ column records whether `pip install agent-framework` installs the workspace pack
| `agent-framework-hyperlight` | `python/packages/hyperlight` | `beta` | `yes` |
| `agent-framework-lab` | `python/packages/lab` | `beta` | `no` |
| `agent-framework-mem0` | `python/packages/mem0` | `beta` | `yes` |
| `agent-framework-milvus` | `python/packages/milvus` | `alpha` | `no` |
| `agent-framework-mistral` | `python/packages/mistral` | `beta` | `yes` |
| `agent-framework-mongodb` | `python/packages/mongodb` | `alpha` | `no` |
| `agent-framework-monty` | `python/packages/monty` | `beta` | `yes` |
Expand Down
21 changes: 21 additions & 0 deletions python/packages/milvus/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) Microsoft Corporation.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
69 changes: 69 additions & 0 deletions python/packages/milvus/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Milvus for Microsoft Agent Framework

`agent-framework-milvus` provides a [Milvus](https://milvus.io/) vector store for the
Microsoft Agent Framework, implementing the framework's `BaseVectorStore` /
`BaseVectorCollection` interfaces. It works alongside the Postgres, Qdrant, Redis and
other vector-store connectors.

## Installation

```bash
pip install agent-framework-milvus
```

This connects to a Milvus server. To use a local `.db` file (Milvus Lite) instead, also
install the Milvus Lite extra, which is published for Linux and macOS only:

```bash
pip install "pymilvus[milvus_lite]"
```

## Usage

```python
from agent_framework import VectorStoreCollectionDefinition, VectorStoreField
from agent_framework_milvus import MilvusStore

definition = VectorStoreCollectionDefinition([
VectorStoreField("key", name="id", type_="int"),
VectorStoreField("data", name="text", type_="str"),
VectorStoreField("vector", name="embedding", dimensions=3, distance_function="cosine_similarity"),
])

# Connects to a Milvus server; set the MILVUS_URI env var or pass uri=...
# For Milvus Lite, pass a local ".db" path as the uri argument, e.g.
# MilvusStore(uri="milvus.db") (requires the pymilvus[milvus_lite] extra). The
# MILVUS_URI env var must be a server address, not a file path.
async with MilvusStore(uri="http://localhost:19530") as store:
collection = store.get_collection(dict, definition=definition, collection_name="docs")
await collection.ensure_collection_exists()
# upsert takes a sequence; pass generate_vectors=False when vectors are supplied.
await collection.upsert([{"id": 1, "text": "hello", "embedding": [0.1, 0.2, 0.3]}], generate_vectors=False)
results = await collection.search(vector=[0.1, 0.2, 0.3], top=5)
async for hit in results: # each hit is a SearchResponse mapping
print(hit["record"], hit["score"])
```

## Configuration

Settings are resolved from explicit arguments, then an optional `.env` file, then
environment variables prefixed with `MILVUS_`:

| Setting | Environment variable | Notes |
| ---------- | -------------------- | -------------------------------------------------- |
| `uri` | `MILVUS_URI` | Server URI (e.g. `http://localhost:19530`). For Milvus Lite, pass a local `.db` path as the `uri=` argument instead — the `MILVUS_URI` env var must be a server address. |
| `token` | `MILVUS_TOKEN` | Access token (API key or `user:password`). |
| `user` | `MILVUS_USER` | Username. |
| `password` | `MILVUS_PASSWORD` | Password. |
| `db_name` | `MILVUS_DB_NAME` | Database name. |

## Notes

- The key field becomes the Milvus primary key (`INT64` for `int` keys, `VARCHAR`
otherwise). Data fields map to typed scalar fields and vector fields to
`FLOAT_VECTOR` fields, each with its own index.
- Distance functions map to Milvus metrics (`cosine_similarity`/`cosine_distance` →
`COSINE`, `dot_prod` → `IP`, `euclidean_distance` → `L2`) and index kinds to Milvus
index types (`hnsw`, `flat`, `ivf_flat`, `disk_ann`, and `default` → `AUTOINDEX`).
- Portable filters support `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `between`, `in`,
`not_in`, and `and`/`or`/`not` groups, translated to Milvus boolean expressions.
14 changes: 14 additions & 0 deletions python/packages/milvus/agent_framework_milvus/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright (c) Microsoft. All rights reserved.

"""Milvus vector stores for Microsoft Agent Framework."""

import importlib.metadata

from ._vector_store import MilvusCollection, MilvusSettings, MilvusStore

try:
__version__ = importlib.metadata.version(__name__)
except importlib.metadata.PackageNotFoundError:
__version__ = "0.0.0"

__all__ = ["MilvusCollection", "MilvusSettings", "MilvusStore", "__version__"]
Loading
Loading