Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = ["static.go"],
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/dependencylimit/static",
visibility = ["//visibility:public"],
deps = ["//submitqueue/extension/speculation/dependencylimit:go_default_library"],
)

go_test(
name = "go_default_test",
srcs = ["static_test.go"],
embed = [":go_default_library"],
deps = [
"@com_github_stretchr_testify//assert:go_default_library",
"@com_github_stretchr_testify//require:go_default_library",
],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Static Dependency Limit

The static `dependencylimit.DependencyLimit` is constructed with a single integer and returns it, unchanged, from every call to `Limit`. It never errors and never consults any external signal — the simplest possible dependency limit, useful for wiring tests, local development, and any queue whose eligibility bound is a fixed operational constant rather than one derived from live capacity or other signals.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package static provides a dependencylimit.DependencyLimit that always
// returns a fixed, construction-time value.
package static

import (
"context"

"github.com/uber/submitqueue/submitqueue/extension/speculation/dependencylimit"
)

// staticLimit is a dependencylimit.DependencyLimit that always returns a
// fixed value.
type staticLimit struct {
// limit is the fixed maximum active-dependency count returned by every
// call to Limit.
limit int
}

// New returns a dependencylimit.DependencyLimit whose Limit always returns
// limit.
func New(limit int) dependencylimit.DependencyLimit {
return staticLimit{limit: limit}
}

// Limit returns the fixed value given to New. It never errors.
func (l staticLimit) Limit(_ context.Context) (int, error) {
return l.limit, nil
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package static

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestLimit_ReturnsConfiguredValue(t *testing.T) {
got, err := New(4).Limit(context.Background())
require.NoError(t, err)
assert.Equal(t, 4, got)
}

func TestLimit_ZeroValue(t *testing.T) {
got, err := New(0).Limit(context.Background())
require.NoError(t, err)
assert.Equal(t, 0, got)
}
23 changes: 23 additions & 0 deletions submitqueue/extension/speculation/enumerator/chain/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = ["chain.go"],
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/enumerator/chain",
visibility = ["//visibility:public"],
deps = [
"//submitqueue/entity:go_default_library",
"//submitqueue/extension/speculation/enumerator:go_default_library",
],
)

go_test(
name = "go_default_test",
srcs = ["chain_test.go"],
embed = [":go_default_library"],
deps = [
"//submitqueue/entity:go_default_library",
"@com_github_stretchr_testify//assert:go_default_library",
"@com_github_stretchr_testify//require:go_default_library",
],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Chain Enumerator

Given a batch and its active dependency batches in arrival order, the chain `enumerator.Enumerator` produces exactly one candidate path: the batch built directly on top of the full chain of those dependencies, in the order given. It never branches, so a batch's tree only ever contains this single path. An empty dependency list still yields one path, whose base is empty — the batch built directly on the target branch.

This is deliberately the least interesting enumerator: it does not weigh which dependencies to include or exclude, does not consider dropping a shaky predecessor, and does not produce a "build alone" alternative alongside the chained one. It is the working, deterministic single-chain baseline — a richer enumerator (for example, one that also offers a build-alone fallback path per dependency) can replace or supplement it without changing the contract.

As with any `enumerator.Enumerator`, the returned path carries structure only — a Base/Head split. The controller assigns its identity, stamps its status, and has the scorer fill its score.
51 changes: 51 additions & 0 deletions submitqueue/extension/speculation/enumerator/chain/chain.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package chain provides an enumerator.Enumerator that enumerates a batch
// as exactly one path, built directly on top of the full ordered chain of
// its active dependencies. It is the single-chain baseline; a multi-path
// enumerator (e.g. one adding build-alone fallback paths) can replace or
// supplement it without changing the contract.
package chain

import (
"context"

"github.com/uber/submitqueue/submitqueue/entity"
"github.com/uber/submitqueue/submitqueue/extension/speculation/enumerator"
)

// chainEnumerator is an enumerator.Enumerator that always enumerates a single
// path per batch.
type chainEnumerator struct{}

// New returns an enumerator.Enumerator that enumerates exactly one path per
// batch: the path's Base is deps in the given order and its Head is the
// batch itself.
func New() enumerator.Enumerator {
return chainEnumerator{}
}

// Enumerate returns exactly one path whose Base preserves deps' order and
// whose Head is the batch — including when deps is empty, in which case the
// single path has an empty Base (the batch builds directly on the target
// branch). The controller owns everything beyond structure: it assigns the
// path its identity, stamps its status, and has the scorer fill its score.
func (chainEnumerator) Enumerate(_ context.Context, batch entity.Batch, deps []entity.Batch) ([]entity.SpeculationPath, error) {
var base []string
for _, dep := range deps {
base = append(base, dep.ID)
}
return []entity.SpeculationPath{{Base: base, Head: batch.ID}}, nil
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package chain

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/uber/submitqueue/submitqueue/entity"
)

func TestEnumerate(t *testing.T) {
tests := []struct {
name string
batchID string
deps []entity.Batch
want []entity.SpeculationPath
}{
{
name: "no deps yields one path with an empty base",
batchID: "q/batch/1",
deps: nil,
want: []entity.SpeculationPath{{Head: "q/batch/1"}},
},
{
name: "deps preserve input order in base",
batchID: "q/batch/4",
deps: []entity.Batch{
{ID: "q/batch/3"},
{ID: "q/batch/1"},
{ID: "q/batch/2"},
},
want: []entity.SpeculationPath{{
Base: []string{"q/batch/3", "q/batch/1", "q/batch/2"},
Head: "q/batch/4",
}},
},
}

e := New()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := e.Enumerate(context.Background(), entity.Batch{ID: tt.batchID}, tt.deps)
require.NoError(t, err)
assert.Equal(t, tt.want, got)
require.Len(t, got, 1)
assert.Equal(t, tt.batchID, got[0].Head)
})
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = ["static.go"],
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/prioritizationlimit/static",
visibility = ["//visibility:public"],
deps = ["//submitqueue/extension/speculation/prioritizationlimit:go_default_library"],
)

go_test(
name = "go_default_test",
srcs = ["static_test.go"],
embed = [":go_default_library"],
deps = [
"@com_github_stretchr_testify//assert:go_default_library",
"@com_github_stretchr_testify//require:go_default_library",
],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Static Prioritization Limit

The static `prioritizationlimit.PrioritizationLimit` is constructed with a single integer and returns it, unchanged, from every call to `Limit`. It never errors and never consults any external signal — the simplest possible prioritization limit, useful for wiring tests, local development, and any queue whose concurrent-build budget is a fixed operational constant rather than one derived from live CI capacity or cost signals.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package static provides a prioritizationlimit.PrioritizationLimit that
// always returns a fixed, construction-time value.
package static

import (
"context"

"github.com/uber/submitqueue/submitqueue/extension/speculation/prioritizationlimit"
)

// staticLimit is a prioritizationlimit.PrioritizationLimit that always
// returns a fixed value.
type staticLimit struct {
// limit is the fixed concurrent-build budget returned by every call to
// Limit.
limit int
}

// New returns a prioritizationlimit.PrioritizationLimit whose Limit always
// returns limit.
func New(limit int) prioritizationlimit.PrioritizationLimit {
return staticLimit{limit: limit}
}

// Limit returns the fixed value given to New. It never errors.
func (l staticLimit) Limit(_ context.Context) (int, error) {
return l.limit, nil
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package static

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func TestLimit_ReturnsConfiguredValue(t *testing.T) {
got, err := New(5).Limit(context.Background())
require.NoError(t, err)
assert.Equal(t, 5, got)
}

func TestLimit_ZeroValue(t *testing.T) {
got, err := New(0).Limit(context.Background())
require.NoError(t, err)
assert.Equal(t, 0, got)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "go_default_library",
srcs = ["sticky.go"],
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/prioritizer/sticky",
visibility = ["//visibility:public"],
deps = [
"//submitqueue/entity:go_default_library",
"//submitqueue/extension/speculation/prioritizationlimit:go_default_library",
"//submitqueue/extension/speculation/prioritizer:go_default_library",
],
)

go_test(
name = "go_default_test",
srcs = ["sticky_test.go"],
embed = [":go_default_library"],
deps = [
"//submitqueue/entity:go_default_library",
"//submitqueue/extension/speculation/prioritizationlimit/fake:go_default_library",
"@com_github_stretchr_testify//assert:go_default_library",
"@com_github_stretchr_testify//require:go_default_library",
],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Sticky Prioritizer

The sticky `prioritizer.Prioritizer` implements sticky build slots: every candidate path that is already `Prioritized` or `Building` holds a slot it keeps until it resolves. It counts those, subtracts the count from the queue's current prioritization limit to get the free budget, and floors the result at zero rather than going negative when the queue is already over budget. It then ranks every `Selected` candidate — the ones asking for a slot — by descending `Score`, and admits as many as the free budget allows with a `Promote` decision each, in ranked order. Ties on `Score` are broken deterministically, first by the path's `Head` batch ID and then by its joined `Base`, so repeated rounds over the same input always order the same way.

Candidates in any other status, and any `Selected` candidate beyond the free budget, are simply omitted — sticky prioritization never emits `Cancel`. This makes it the least disruptive prioritization policy: a lower-scored path that is already running is never evicted to make room for a newer, higher-scored one; the newer path instead waits until a slot frees up naturally. It is constructed with its `prioritizationlimit.PrioritizationLimit`, consulting it fresh on every call since the budget can change between rounds.
Loading
Loading