feat: add archive infrastructure for EOL distribution packages#47
Open
abtreece wants to merge 8 commits into
Open
feat: add archive infrastructure for EOL distribution packages#47abtreece wants to merge 8 commits into
abtreece wants to merge 8 commits into
Conversation
5 tasks
The previous code degraded any non-2xx response (auth errors, 5xx, redirects) to LATEST_VERSION=0 whenever the suffix indicated an archive bucket. That silently turns a transient outage or misconfiguration into Caddy redirecting clients to /versions/0/... 404s — broken behavior masquerading as valid config. Only treat HTTP 404 as the legitimate "archive not yet populated" case; surface every other failure.
noahssarcastic
previously approved these changes
May 19, 2026
noahssarcastic
left a comment
noahssarcastic
left a comment
Collaborator
There was a problem hiding this comment.
Looks good, reviewed in tandem with fullstaq-ruby/server-edition#191.
FooBarWidget
previously approved these changes
May 21, 2026
FooBarWidget
left a comment
FooBarWidget
left a comment
Member
There was a problem hiding this comment.
Also looks good to me, agree with @noahssarcastic: query_repo_version could use some code clarity improvements but not blocking.
Also, the infra overview docs need to be updated. Since #57 is not yet merged, I suggest updating #57 with the changes made in this PR #47.
Feel free to deploy @abtreece.
Documents the new APT/YUM archive infrastructure introduced in this PR: the two new public-read GCS buckets, the deliberate absence of CI write access (frozen-mirror invariant enforced in IAM), the Azure DNS zones and apex NS delegation for the archive subdomains, and the 404 fallback behavior in query-latest-repo-versions.rb that lets the web server start cleanly before the first migration runs. Addresses FooBarWidget's note on PR fullstaq-ruby#57 that fullstaq-ruby#47's changes should ship with their own documentation.
abtreece
dismissed stale reviews from FooBarWidget and noahssarcastic
via
b58dae6
Address review feedback on PR fullstaq-ruby#47 from @noahssarcastic and @FooBarWidget: the awkward `suffix:` parameter in query-latest-repo-versions.rb was a downstream symptom of inconsistent bucket naming (`-repo-archive` broke the `-repo` terminal-suffix convention used by the live buckets). Rename: <prefix>-server-edition-apt-repo-archive -> -apt-archive-repo <prefix>-server-edition-yum-repo-archive -> -yum-archive-repo Safe to do now -- the buckets don't exist yet (PR not deployed). With consistent naming, query_repo_version becomes a direct lookup: `type` is the bucket-name slot, and the only special-casing for archives is a semantic `allow_missing:` flag for the pre-migration bootstrap window. No string manipulation, no latent type/suffix mismatch footgun. Coordination: server-edition#191's CI workflow sets ARCHIVE_REPO_BUCKET_NAME -- that value will need to match the new bucket name before the first migration runs.
Collaborator
Author
|
For reviewers tracking both PRs: c1814eb renames the archive buckets from |
Collaborator
Author
@FooBarWidget I've been deep in some home repair things this past week. I'll try to get it deployed this weekend. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
apt-archive.fullstaqruby.organdyum-archive.fullstaqruby.orgquery-latest-repo-versions.rbto query archive bucket versions, with graceful fallback (version 0) before first migrationContext
Addresses fullstaq-ruby/server-edition#190 — CI disk space exhaustion from the growing Aptly state archive.
Archive repos follow the PostgreSQL (
apt-archive.postgresql.org) and HashiCorp (archive.releases.hashicorp.com) pattern: frozen repositories for EOL distributions, served as static redirects to GCS.Companion PR: fullstaq-ruby/server-edition#191 (migration scripts, build config, runbook)
Implementation plan:
context/plans/EOL-ARCHIVE-MIGRATION.md(Tasks 1-3)Design notes
apt/yumbuckets grantroles/storage.objectAdminto thegithub-ci-deployworkload identity; the archive buckets deliberately omit this binding. Migration happens out-of-band via the companionserver-edition#191scripts, and the frozen-mirror invariant is enforced in IAM, not by convention.azurerm_dns_zonewith an NS record in the apex zone. Caddy's ACME DNS-01 challenge for the new subdomains will fail until the apex NS records propagate, so the Caddy restart in step 3 must happen after propagation completes (verified by step 2'sdigcheck), not concurrent withterraform apply.force_destroy = trueon archive buckets is inherited from the existing bucket pattern and applies here as well. Worth flagging since the data is by definition irreplaceable EOL package archives — accepting the risk to stay consistent with the surrounding module, but reviewers should confirm this is the intended trade-off.Deployment sequence
terraform applyinterraform/to create buckets and DNSdigper test plan)Test plan
terraform init -backend=false && terraform validate && terraform fmt -check -diff -recursivepassesdig apt-archive.fullstaqruby.org Aanddig yum-archive.fullstaqruby.org Areturn backend server IPcurl -I https://apt-archive.fullstaqruby.org/dists/returns 302 to GCS