CI builds were failing at random with:
buildx failed with: toomanyrequests: Rate exceeded
The 429 comes from public.ecr.aws, not Docker Hub. AWS caps unauthenticated
ECR Public pulls at 1 per second per source IP (authenticated: 10/s). The
build matrix starts 11 jobs at once and each resolves 4 base images, so
roughly 44 anonymous pulls land in a couple of seconds — from GitHub runner
IPs that are shared with every other GitHub customer.
Measured against public.ecr.aws with an anonymous token, 60 requests at
concurrency 30 returned 31x 429 in 0.48s.
The same probe against mirror.gcr.io (Google's Docker Hub pull-through
cache) returned zero errors: 750 manifest requests up to ~141 req/s, plus
120 layer blob requests at concurrency 60. Google publishes no rate limit
for it, so this is measured headroom, not a contract — but it is roughly
10x the pipeline's peak rate, and cached pulls do not count against Docker
Hub's limits either.
Authenticating to ECR Public was the alternative. It was rejected because
10 pulls/s is still under the ~44-pull burst, it needs an AWS account plus
a secret, and secrets never reach fork pull requests — so forks would keep
failing. The mirror fixes forks too.
Verified buildkit honours the mirror block by routing a build through a
local logging registry: all 6 requests (manifests and blobs) hit the mirror,
none went to Docker Hub directly. Confirmed mirror.gcr.io answers 200 for
buildkit's "?ns=docker.io" query form on all four images, for both GET and
HEAD. Confirmed buildkit falls back to Docker Hub when the mirror is
unreachable — the build still succeeds, but the resolve takes ~30s instead
of ~0.3s, so a mirror outage means slow builds, not broken ones. The
existing Docker Hub login covers that fallback on main-repo runs.
msitools.dockerfile is only used by the local `make docker-msi` target, but
is switched over too so no ECR Public reference is left behind.
* chore(ci): update GitHub Actions to latest major versions
Update actions/cache v4→v5, actions/github-script v3→v7,
actions/stale v9→v10, docker/login-action v3→v4,
docker/setup-buildx-action v3→v4, and docker/metadata-action v5→v6.
The github-script upgrade also migrates Octokit API calls from
github.* to github.rest.* namespace (required since v5).
* fix(ci): address review feedback on GitHub Actions update
Pass github_token to docker/metadata-action@v6 to avoid API rate
limiting. Fix github-script pagination to use the correct Octokit
paginate.iterator pattern (pass endpoint method, not awaited response).
* refactor(build): remove CPP taglib adapter
Remove the CGO-based TagLib adapter (adapters/taglib/) and all
cross-taglib build infrastructure. The WASM-based go-taglib adapter
(adapters/gotaglib/) is now the sole metadata extractor.
- Delete adapters/taglib/ (CPP/CGO wrapper)
- Delete .github/actions/download-taglib/
- Remove CROSS_TAGLIB_VERSION, CGO_CFLAGS_ALLOW, and all taglib-related
references from Dockerfile, Makefile, CI pipeline, and devcontainer
* fix(scanner): gracefully fallback to default extractor instead of crashing
Replace log.Fatal with a graceful fallback when the configured scanner
extractor is not found. Instead of terminating the process, the code now
warns and falls back to the default taglib extractor using the existing
consts.DefaultScannerExtractor constant. A fatal log is retained only for
the case where the default extractor itself is not registered, which
indicates a broken build.
* test(scanner): cover default extractor fallback and suppress redundant warn
Address review feedback on the extractor fallback in newLocalStorage:
- Only log the "using default" warning when the configured extractor
differs from the default, so a broken build (default extractor itself
missing) logs only the fatal — not a misleading "falling back" warn
followed immediately by the fatal.
- Add a unit test that registers a mock under consts.DefaultScannerExtractor,
sets the configured extractor to an unknown name, and asserts the local
storage is constructed using the default extractor's constructor.
* build: new pipeline, new way to cross-compile and build docker images locally. (#3383)
* build: use alternative repositories
* build: fix
* build: validate taglib downloads
* build: control concurrency
* build: validate xx version
* build: remove taglib download validation as the version can be changed as an argument.
