Skip to content

EPMDEDP-17188: feat: Add stale branch detection and cleanup for codebase branches#296

Merged
SergK merged 1 commit into
masterfrom
add-support-for-stale-branch-reconcile
Jul 12, 2026
Merged

EPMDEDP-17188: feat: Add stale branch detection and cleanup for codebase branches#296
SergK merged 1 commit into
masterfrom
add-support-for-stale-branch-reconcile

Conversation

@SergK

@SergK SergK commented Jul 12, 2026

Copy link
Copy Markdown
Member

Periodically verify that every codebase branch still exists in the git repository, mark missing ones and optionally clean them up, so platforms with many branches do not accumulate CodebaseBranch resources for branches deleted from git.

Detection (always on, configurable):

  • list remote branches without cloning (git ls-remote) via the git client
  • leader-elected periodic checker (BRANCH_STALE_CHECK_INTERVAL, default 24h, 0 disables) doing one listing per codebase repository
  • record the result as a Stale status condition (source of truth), mirror it with the app.edp.epam.com/stale label for selector-based filtering and emit events
  • never mark branches on git connectivity or authorization errors; skip default, not-yet-pushed and terminating branches
  • preserve status conditions in every CodebaseBranch status rewrite so the Stale condition survives regular reconciles of the branch

Cleanup (opt-in per Codebase via app.edp.epam.com/branch-cleanup-strategy=auto):

  • delete stale branches not referenced by any CDPipeline inputDockerStreams or Stage autotest quality gate; the owned CodebaseImageStream is garbage-collected
  • keep referenced branches marked, recording the retaining resource in the condition message and a StaleBranchRetained event

Protection:

  • the admission webhook now rejects deleting a CodebaseBranch that participates in a deployment (previously enforced only client-side by the portal), except during Codebase cascade deletion
  • works on clusters without cd-pipeline-operator installed

Fix: release the CodebaseBranch finalizer when the owning Codebase is gone. When a Codebase is deleted, its branches are removed afterwards by garbage collection, at which point the owning Codebase no longer exists. Reconcile failed on the Codebase lookup before reaching the deletion flow, so such branches stayed terminating forever with their finalizer never released. Run the deletion flow directly when the branch is terminating and its Codebase is not found, letting cascade deletion complete.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Enhancement (non-breaking change which improves an existing feature or documentation)
  • Breaking change (fix or feature that would cause existing functionality not to work as expected)

How Has This Been Tested?

local e2e test

Checklist:

  • I have performed a self-review of my code
  • I have commented on my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Pull Request contains one commit. I squash my commits.

@SergK SergK requested a review from a team as a code owner July 12, 2026 09:57
…ase branches

Periodically verify that every codebase branch still exists in the git repository,
mark missing ones and optionally clean them up, so platforms with many branches do
not accumulate CodebaseBranch resources for branches deleted from git.

Detection (always on, configurable):
- list remote branches without cloning (git ls-remote) via the git client
- leader-elected periodic checker (BRANCH_STALE_CHECK_INTERVAL, default 24h,
  0 disables) doing one listing per codebase repository
- record the result as a Stale status condition (source of truth), mirror it with
  the app.edp.epam.com/stale label for selector-based filtering and emit events
- never mark branches on git connectivity or authorization errors; skip default,
  not-yet-pushed and terminating branches
- preserve status conditions in every CodebaseBranch status rewrite so the Stale
  condition survives regular reconciles of the branch

Cleanup (opt-in per Codebase via app.edp.epam.com/branch-cleanup-strategy=auto):
- delete stale branches not referenced by any CDPipeline inputDockerStreams or
  Stage autotest quality gate; the owned CodebaseImageStream is garbage-collected
- keep referenced branches marked, recording the retaining resource in the
  condition message and a StaleBranchRetained event

Protection:
- the admission webhook now rejects deleting a CodebaseBranch that participates
  in a deployment (previously enforced only client-side by the portal), except
  during Codebase cascade deletion
- works on clusters without cd-pipeline-operator installed

Fix: release the CodebaseBranch finalizer when the owning Codebase is gone.
When a Codebase is deleted, its branches are removed afterwards by garbage
collection, at which point the owning Codebase no longer exists. Reconcile
failed on the Codebase lookup before reaching the deletion flow, so such
branches stayed terminating forever with their finalizer never released.
Run the deletion flow directly when the branch is terminating and its
Codebase is not found, letting cascade deletion complete.

Signed-off-by: Sergiy Kulanov <sergiy_kulanov@epam.com>
@SergK SergK force-pushed the add-support-for-stale-branch-reconcile branch from a22cdb1 to 77606df Compare July 12, 2026 10:15
@SergK SergK merged commit 9860ad8 into master Jul 12, 2026
6 checks passed
@SergK SergK deleted the add-support-for-stale-branch-reconcile branch July 12, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant