feat(pins): add ipfs_pin_count tool and limit/offset pagination for ipfs_list_pins - #15
Open
karawitan wants to merge 1 commit into
Open
feat(pins): add ipfs_pin_count tool and limit/offset pagination for ipfs_list_pins#15karawitan wants to merge 1 commit into
karawitan wants to merge 1 commit into
Conversation
…pfs_list_pins
Nodes with large pinsets (hundreds of thousands to millions of pins) make
the current ipfs_list_pins tool impractical: it materializes the full
pinset and returns every CID through the MCP channel, blowing up context
and timing out.
- Add core countPins(): streams Kubo `pin/ls --type=all --stream` and
tallies counts by type (direct/recursive/indirect/total) with constant
memory — no CIDs are accumulated.
- Add listPins({limit, offset}) pagination; when limit is set, iteration
stops early instead of draining the stream.
- Expose both on the Meshkit facade (primary node) and the fil.one/S3
client (count = stored objects, pagination via slice).
- MCP: new ipfs_pin_count tool; ipfs_list_pins now accepts optional
limit/offset and returns count metadata.
Generated with [Devin](https://devin.ai)
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Nodes with large pinsets (hundreds of thousands to millions of pins) make the current
ipfs_list_pinstool impractical: it materializes the full pinset and returns every CID through the MCP channel — huge memory spikes, tool timeouts, and blown agent context.ipfs_pin_countMCP tool — streams Kubopin/ls --type=all --streamand returns only counts by type (direct,recursive,indirect,total). Constant memory: no CIDs are accumulated, safe for pinsets of any size.ipfs_list_pinspagination — new optionallimit/offsetparams. Whenlimitis set, iteration stops early instead of draining the stream. Response now includescount(andlimit/offsetwhen given).MeshkitClient.countPins(): Promise<PinCount>(newPinCounttype)MeshkitClient.listPins(options?: ListPinsOptions)Meshkit.countPins()/Meshkit.listPins(options)(primary node)countPins= stored object count, pagination via slicecountPinsViaRpc/applyPinLsLinehelpers from@ipfs-meshkit/coreImplementation notes
countPinsViaRpcPOSTs/api/v0/pin/ls?type=all&stream=trueand tallies the NDJSON stream line-by-line (handles the streamed{"Cid":...,"Type":...}format and the legacyKeysmapping; the legacy type-lessPinsarray is ignored). Optional authheadersfrom the node config are forwarded.listPins({limit, offset})on Kubo skipsoffsetentries then stops the async iterator as soon aslimitCIDs are collected.Test plan
vitest run --project unit), including new tests:pin-count.test.ts(line tallying, chunk-boundary splits, legacy formats, HTTP errors),create-client.test.ts(early-stop pagination, offset skip, count via client),meshkit.test.ts(primary-node-only, option passthrough),storage.test.ts(MCP handlers)tsc --noEmitclean;npm run build+ mcp build succeedipfs_list_pins {limit:2, offset:1}returns instantly;ipfs_pin_countstreams without unbounded memory growthGenerated with Devin