Repository navigation
Map each Go package, method and variable to one JavaType - #8970
Merged
Merged
Conversation
The Go type mapper built a new JavaType for a package at every reference to it (the `assert` in `assert.Equal`, the declaring type of a package-level function, the owner of a package-level variable), for `go.tuple` at every multi-value result, and a new method or variable type at every use. The RPC ref table interns by identity, so none of those was ever a ref hit. Parsing milvus-io/milvus, 945 files in, the Java side held 488,018 JavaType.Class instances, among them 74,802 `go.tuple`, 47,988 `github.com/stretchr/testify/assert` and 37,153 `context`. The mapper now caches them: packages by path, methods by *types.Func, and variables by *types.Var and the scope they were read in. ParseProject also keeps one parser, and so one mapper, per module: its packages share the module's ProjectImporter, which caches each imported *types.Package, so a type imported by many packages is the same object in each.
This was referenced Sep 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Go type mapper built a new
JavaTypefor a package at every reference to it: theassertinassert.Equal, the declaring type of a package-level function, the owner of a package-level variable. It did the same forgo.tupleat every multi-value result, and built a new method or variable type at every use. The RPC ref table interns by identity, so none of those copies was ever a ref hit, and every one of them crossed the wire and stayed interned on both sides.Parsing
milvus-io/milvus(3,635 Go files), 945 files in, the Java side held 488,018JavaType.Classinstances. The most-duplicated names were packages and tuples:go.tuplegithub.com/stretchr/testify/assertcontextgithub.com/milvus-io/milvus/pkg/v3/mlogbuiltinChange
The mapper caches packages by path (which also covers
go.tupleandbuiltin), methods by*types.Func, and variables by*types.Vartogether with the scope they were read in, since a local's owner is its enclosing function. Nothing mutates a method or variable type aftermapSignaturebuilds it, so sharing the instances is safe.ParseProjectalso keeps one parser, and so one mapper, per module rather than per package. A module's packages already share itsProjectImporter, which caches each imported*types.Package, so a type imported by many packages is the same object in each and now maps to oneJavaType. On its own this barely moved the numbers above; the per-use caching is what matters.Result
With #8512 keeping refs across files and #8969 dropping each tree once the caller is done with it,
milvus-io/milvusparses end to end with a 1 GB heap, which previously ran out of memory. Refs averaged about 1,100 per file instead of about 2,600, ending at 3.94M for the project.This PR doesn't depend on the other two and is useful on its own: the Go peer builds far fewer objects, and anything that holds the trees holds far fewer types.