EPMDEDP-17188: feat: Add stale branch detection and cleanup for codebase branches#296
Merged
Merged
Conversation
…ase branches Periodically verify that every codebase branch still exists in the git repository, mark missing ones and optionally clean them up, so platforms with many branches do not accumulate CodebaseBranch resources for branches deleted from git. Detection (always on, configurable): - list remote branches without cloning (git ls-remote) via the git client - leader-elected periodic checker (BRANCH_STALE_CHECK_INTERVAL, default 24h, 0 disables) doing one listing per codebase repository - record the result as a Stale status condition (source of truth), mirror it with the app.edp.epam.com/stale label for selector-based filtering and emit events - never mark branches on git connectivity or authorization errors; skip default, not-yet-pushed and terminating branches - preserve status conditions in every CodebaseBranch status rewrite so the Stale condition survives regular reconciles of the branch Cleanup (opt-in per Codebase via app.edp.epam.com/branch-cleanup-strategy=auto): - delete stale branches not referenced by any CDPipeline inputDockerStreams or Stage autotest quality gate; the owned CodebaseImageStream is garbage-collected - keep referenced branches marked, recording the retaining resource in the condition message and a StaleBranchRetained event Protection: - the admission webhook now rejects deleting a CodebaseBranch that participates in a deployment (previously enforced only client-side by the portal), except during Codebase cascade deletion - works on clusters without cd-pipeline-operator installed Fix: release the CodebaseBranch finalizer when the owning Codebase is gone. When a Codebase is deleted, its branches are removed afterwards by garbage collection, at which point the owning Codebase no longer exists. Reconcile failed on the Codebase lookup before reaching the deletion flow, so such branches stayed terminating forever with their finalizer never released. Run the deletion flow directly when the branch is terminating and its Codebase is not found, letting cascade deletion complete. Signed-off-by: Sergiy Kulanov <sergiy_kulanov@epam.com>
a22cdb1 to
77606df
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Periodically verify that every codebase branch still exists in the git repository, mark missing ones and optionally clean them up, so platforms with many branches do not accumulate CodebaseBranch resources for branches deleted from git.
Detection (always on, configurable):
Cleanup (opt-in per Codebase via app.edp.epam.com/branch-cleanup-strategy=auto):
Protection:
Fix: release the CodebaseBranch finalizer when the owning Codebase is gone. When a Codebase is deleted, its branches are removed afterwards by garbage collection, at which point the owning Codebase no longer exists. Reconcile failed on the Codebase lookup before reaching the deletion flow, so such branches stayed terminating forever with their finalizer never released. Run the deletion flow directly when the branch is terminating and its Codebase is not found, letting cascade deletion complete.
Type of change
How Has This Been Tested?
local e2e test
Checklist: