Export: trimming enhancements - #3530

Merged

peterebden merged 155 commits into

thought-machine:masterfrom

DuBento:export-from-package-metadata

Sep 9, 2026

Commits on Aug 28, 2026

Package level metadata that includes a mapping of build statements to build targets
duarte
committed
8d0ecc4
export using BuildFileMetadata logic
duarte
committed
c91668c
register caller scope for callstack-like traversal
duarte
committed
1276403
bufio writer for build statements
duarte
committed
836fff5
rewrite export logic into explicit flow
duarte
committed
a505981
export targets related to build statement
duarte
committed
d3e67bc
enrich target with subincludes by looping though all scopes
duarte
committed
d09e7d0
separate buildstmt register from adding target
duarte
committed
1fb4f18
select and write subincludes
duarte
committed
b0c3c1e
skip statement for preloaded subincludes
duarte
committed
3abc3cd
test: trim subincludes
duarte
committed
68b61e0
rename file
duarte
committed
aeb2e4a
use slices.collect and sort interface
duarte
committed
1966979
rewrite export with 2 concrete interfaces: default and notrim
duarte
committed
50110d3
notrim: full copy BUILD file and visit sources
duarte
committed
65fd873
double new lines between targets
duarte
committed
82f9587
suppress diff output when enforcing repo differences
duarte
committed
00f617f
export by filtering the original BUILD file
duarte
committed
47ab394
subincludes use label short string
duarte
committed
a2aa6a0
Simplify package filtering method into a more explicit "switch" case
duarte
committed
28f61c6
skip subrepos and internal packages when writing package file
duarte
committed
6616b6e
fix reusing err var resulted in failed filtering
duarte
committed
f849a78
doc strings for package metadata
duarte
committed
7fffc39
export missing doc strings
duarte
committed
b1fc186
test: named go_repo and change testify for slimmer UUID
duarte
committed
6d572c7
export dependencies of subrepos - 13/14 tests passing
duarte
committed
ec888c5
test: internal repo test using temp directory to avoid stale data
duarte
committed
a4da492
fix: 0 label subinclude
duarte
committed
e0fcfe2
adjusting dependency lookup and adjacent target test to include secondary build def with sources
duarte
committed
7eda2cb
test: add custom tool to native test
duarte
committed
9736002
test: go_test export with several deps
duarte
committed
2f0a275
optional metadata parsing
duarte
committed
88b5205
test: minimal subinclude statement
duarte
committed
4633d4f
collect map keys on active subinclude labels
duarte
committed
2b2af8f
non-fatal warning for missing source while exporting
duarte
committed
e0f2b61
skip internal package export
duarte
committed
4b984b6
move some fatal to error and continue
duarte
committed
5faf4c1
missing docstrings
duarte
committed
942c4b2
run go fmt and plz fmt
duarte
committed
215adc0
move to error and continue for target lookup
duarte
committed
4746de6
rename new parser method on test files
duarte
committed
1a758d0
use pkg.Metadata directly and remove intermediate pkg methods
DuBento
committed
e5ba148
mutex in packagemetadata
DuBento
committed
31d4afe
update stmt provider to avoid dereference
DuBento
committed
92c2e17
NewPackage with variadic optional functions
DuBento
committed
8733b00
infof to debugf
DuBento
committed
488f951
package metadata doc comments improvements
DuBento
committed
26d9a46
improve doc comments and adjust method visibility for export.go
DuBento
committed
38ffa62
hide build output when testing export
DuBento
committed
8da1e96
open and write of exported package file merged into the same method
DuBento
committed
21e23c6
update export_test.go with suggestions
DuBento
committed
2377776
apply review suggestions to export.go
DuBento
committed
d05a9c3
rename and doc fields for scope
DuBento
committed
6bc28b5
revert new line change of target
DuBento
committed
c1a04df
set builtin code filename for scope for debugging
DuBento
committed
a95784c
Rework packagemetadata to return empty slices or nil instead of errors. Add fatal to no-op implementation
DuBento
committed
1d749b7
linter fix imports
DuBento
committed
58e6532
ensure subrepo target exists. Failing to export //... on this repo
DuBento
committed
db93587
go test for filtered package
DuBento
committed
bea7e98
Add doc comments for providers in asp.interpreter
DuBento
committed
2d608dd
test: function def in package file
DuBento
committed
4c81a60
unit testing current statement and active subincludes
DuBento
committed
165045d
remove nested maps and pointers in favor of using build labels
DuBento
committed
beee484
unexport noTrim implementation
DuBento
committed
5ee2bcd
AST based trimming (if-else and for):
DuBento
committed
3370418
handle subinclude variables by tracking origin and used.
DuBento
committed
029de6a
scope metadata interface for optional metadata processing
DuBento
committed
ac76bdd
test: adjust interpreter tests with scope metadata and origin tracking changes
DuBento
committed
fa25ce2
golint trimmer
DuBento
committed
69e310b
fix missing targets by keeping parser running and parsing inline during export
DuBento
committed
0fe5d00
split export implementations into different files
DuBento
committed
6cdcebc
fix: propagate cursor
DuBento
committed
1651a3e
simplify calls to getOrParseTarget
DuBento
committed
fda8cd9
remove use of formatter and trim newlines
DuBento
committed
fd68740
reuse parser between build files
DuBento
committed
58d8936
use of baseExporter directly and interface cleanup
DuBento
committed
74d290a
interpreter preloads info prevents secondary loading/subincludes
DuBento
committed
e2ef287
fix: deadlocking on closed results thread
DuBento
committed
def5006
test: use system cache for speeding up e2e tests
DuBento
committed
8d95ab5
add explicit targetToStmt mapping to improve CPU performance on FindStatement and RelatedTargets methods
DuBento
committed
6857d4d
use cmap in package metadata, removing explicit usage of a shared mutex for all maps
DuBento
committed
3f2427e
docs: interface implementation comment
DuBento
committed
a44e729
rename exporter source files
DuBento
committed
56bc568
apply feedback from multiple comments
DuBento
committed
d11e014
local subrepos: fixes (relative filenames) and test
DuBento
committed
ebe361d
export transitive subincludes, fixing unexported targets subincluded by build_defs + test cases
DuBento
committed
0500a75
export gitignore
DuBento
committed
bb6b0e5
test: glob for sources
DuBento
committed
ee374ce
Package parsing via WaitForPackage
DuBento
committed
d87189e
test: move trim test files into data
DuBento
committed
97e90d7
test: trim subinclude in third_party/go test
DuBento
committed
89aca5c
export label set
DuBento
committed
94fd517
track used objects as a stack
DuBento
committed
add2812
scopeMetadata: private, rename and comments
DuBento
committed
a087c75
rename active to required subincludes
DuBento
committed
c459832
imp: skip scope and package metadata tracking for external subrepos
DuBento
committed
24035dc
remove doc reference to required target naming for successful export
DuBento
committed
41c72f9
Only pop symbols from stack in package scope + rework newScopeMetadata to also avoid tracking inside subincludes
DuBento
committed
d819246
lazy initialize symbol origins
DuBento
committed
348cdc7
Close package wait channels on error. Introduced a deadlock when KeepParser is set.
DuBento
committed
d75b5ea
Support method calls as arguments by popping the symbol stack only at package level
DuBento
committed
137f012
refactor: record all statements and respective origins for all statements (including non-target generators).
DuBento
committed
efcaabb
Package level scope metadata; Tracking done at package level only.
DuBento
committed
8332c5f
Package metadata lookup table in interpreter to avoid recursive caller traversal.
DuBento
committed
46d8b29
Trimmer: always keep for expression but trim stmts inside
DuBento
committed
99f5b29
glob statement: register files required by statements
DuBento
committed
fa7847f
Checkpoint and restore symbol and file stack to support different calls to interpretStatements with the package level scope (e.g. for loop)
DuBento
committed
3ea8dec
move to BuildLabels in export and metadata to avoid pressure on GC
DuBento
committed
54b70c6
track interpreted statements (mark as 0 targets) to correctly identify if blocks that have interpreted stmts (var redeclaration)
DuBento
committed
b38e018
subinclude tracking for dynamic subincludes
DuBento
committed
a39676f
track inner level globs.
DuBento
committed
6e40952
feat(cmd): query print metadata - print package metadata
DuBento
committed
beff770
track inner statements (e.g. if clauses) and trim clauses based on if they were interpreted oir not
DuBento
committed
4db88e3
lint suggestions: remove hashBuildTarget and initialise labels slice with cap
DuBento
committed
a935994
feat(cmd): metadata json formatting
DuBento
committed
376d14b
progress monitor for export
DuBento
committed
08b8e94
metadata: drop json printing and optional hidden targets
DuBento
committed
ee07ae8
maintain subinclude label order
DuBento
committed
3ced65d
combined build inputs
duarte
committed
433333e
Pseudo uniform distribution
duarte
committed
4253c25
Package metadata renaming and doc comment fixes
duarte
committed
b7ad615
Justify subsequent calls to RegisterStatement
duarte
committed
e924bf4
Address package metadata review comments: docs, zero value and sort
duarte
committed
397ed51
Refactor: consolidate maps into single statement map with statement metadata
duarte
committed
31361d7
Refactor: remove subinclude statement mapping and use isSubincludeStatement bool instead
duarte
committed
9f5ad5e
State: test for logBuildError
duarte
committed
2c1644c
export test formatting
duarte
committed
beef2ac
Fix if trim test: declared interpreted statements.
duarte
committed
30a5ee5
getOrParse safety timeout
duarte
committed
777b1f1
fix infinite counter
duarte
committed
5632e13
address query.metadata comments
duarte
committed
3bf3179
strip symbols
duarte
committed
0931003
s/CleanUp/Cleanup
duarte
committed
1cbfd18
s/unneded/unneeded
duarte
committed
e6ff712
standardise use of "unused" in tests
duarte
committed
ab48c30
comments for interpreter.go
duarte
committed
7f0bdfc
test: target variable trimming
duarte
committed
a081324
revert: track preloaded in interpreter
duarte
committed
229ea4f
fix: add new lines at the end of files
duarte
committed
10b3f05
revert one time variable
duarte
committed
23b4f99
test: move complex go test to e2e
duarte
committed
7ba7f18
lint: plz fmt
duarte
committed
585d95a
fix: wrong reference to unused lib in e2e test
duarte
committed
d32f1e6
refactor/revert: resolve package targets upfront instead of keeping parser alive
duarte
committed
90e7436
refactor: decouple buildStatement from core by implementing interface in ASP
duarte
committed
7552be7
go lint
duarte
committed
ef3bd43

Commits on Sep 2, 2026

unexport labelset
duarte
committed
9b8ca27
refactor: use RWmutex instead of cmaps for package metadata
duarte
committed
4f7c136
refactor: introduce struct TrackedStatement instead of sorted slices for query metadata
duarte
committed
0f073e4

Commits on Sep 3, 2026

test: trimming of variables that include targets
duarte
committed
0ae4949
test: refactor build error test for better consistency
duarte
committed
2c4ae9f
refactor: subincludeArgs as iterator
duarte
committed
ff407e4

Commits on Sep 8, 2026

typos and unused code
duarte
committed
7e37fb3
fix: graciously skip system dependencies when exporting sources
duarte
committed
08f6d86
doc: readme on the export process
duarte
committed
341397f