[tango]: Add integration testing#156
Conversation
| // 1f2e3e9 Honor OutputConfig include_hashes/include_tags/include_attributes (#116) | ||
| // The second commit adds controller/output_filter.go and output_filter_test.go. | ||
| const firstSHA = "046de2c20b5492cd5606d32fd632a38b8b70c8f6" | ||
| const secondSHA = "1f2e3e9245b159006cf2103becd51c5c1b6ec868" |
There was a problem hiding this comment.
can this be in Makefile envs instead?
There was a problem hiding this comment.
Not sure it makes much sense to do that. These test cases are asserting on expected changed targets for a specific set of SHAs. The expectations are hard-coded, so you can't really change the SHAs using env vars or the test will just.. fail. I think it makes more sense to keep the input next to the expected output.
I think a potential extension eventually would be to move all the shas & expected output into some sort of testdata files rather than hard-coding them, but this is already long-running as it is so not sure it matters until we can support running a lot of these tests somehow.
|
@sywhang mentioned the integ test can be done in another approach. Could you take a look? |
That's gonna take a while since we need to write various utils for emulating the git repo workload and do some fancy stuff. Let's check in at least one E2E test that we can use, since we're about to do some big refactoring. |
Add an integration test to tango. This test: * Spins up the tango server, configured with the tango repo itself as the target * Creates a client connected to it * Calls its APIs (GetTargetGraph and GetChangedTargets) * Asserts some responses from the server against expected data for actual commits in the tango repository. This provides a useful signal for how well tango works as an entire system. In a future PR, we can also re-use the same testing components to write benchmarks that measure tango's latency.
JacobOaks
force-pushed
the
joaks/e2e-test
branch
from
329c1bc to
e8009df
Compare
3 participants
Add some integration testing to tango. These tests:
I decided to use the actual tango repository to provide a heightened realism to the tests.
I also prototype a setup that generated an entire repository with multiple commits at runtime,
but just the boilerplate to bring in rules_go/other basics to support any non-contrived examples
seems to complicate the test code enough that I figured I might as well just use tango itself.
The downside of this approach is that I couldn't easily really assert against any actual piece of the graph
in entirety in testing GetTargetGraph, because it's pretty huge in a real repository with external dependencies. Instead, I assert against some specific expected edges.
This provides a useful signal for how well tango works as an entire system. In a future PR, we can also re-use the same testing components to write benchmarks that measure tango's latency.
This also adds a separate make target and CI step to run this since, in local testing, it can take about a minute or so.