feat(stesol-576-590): Enable BlackDuck Policy Gating + Scan build-inputs & embedding ghcr images

[NeethuESim]

PR Description

This PR covers changes for STESOL-576 (Implement Policy gating) & STESOL-590 (Schedule Blackduck Scan for GHCR build-input and embedding images)

• Release policy is defined with scan failing when Blocker/Critical issues are found via blackducksca_scan_failure_severities: 'BLOCKER,CRITICAL' tags. Currently it is setup for license & security issues. Unable to test the license only blocking because of blackduck server issues.

• All issues are tracked via Arm BlackDuck UI dashboard - https://arm.app.blackduck.com/api/projects/53e38e5f-dafd-4b59-98cd-b73c2455d09e

• Policy violation is also set in Arm BlackDuck UI dashboard using tag applyhighvulnspolicyasblocker

• Since the build-embeddings generates a scratch image with data-only artifacts rather than runnable containers, Blackduck finds no components/vulnerabilities. For more vulnerability coverage, embedding-toolchain is also scanned.

• Scan is setup for following versions under Arm:MCP project -
  1.0 - Codebase Source Scan
  mcp-build-inputs-container-amd64-1.0 - GHCR build-inputs for amd64
  mcp-build-inputs-container-arm64-1.0 - GHCR build-inputs for arm64
  mcp-embedding-vectorstore-1.0 - GHCR embedding-vectorstore
  mcp-embedding-generator-container-1.0 - GHCR embedding-generator-container

• SBOM reports for all these can be generated & downloaded from the UI in CycloneDX format.

• Dry run is added for testing purpose. We can keep it to debug BlackDuck issues in future or remove it. I am good with either.

• Automatic main or scheduled runs publish only after successful scans.

• Scans for the source code is triggered through black-duck-security-scan-ci.yml where as scan for build-inputs & embedding images is through action.yml invoked from build-mcp-inputs.yml, build-embeddings.yml & build-embedding-toolchain.yml. Two flows are kept separate since they scan different targets.

- Currently BlackDuck Scans are failing due to timeout issue with their servers. I have reached out to their team via slack.

Scan Architecture & Flow

Repository
├-- Source scan
│ └─ Black Duck source/SCA analysis
│
└─ Container build workflows
├-- MCP build-input images: AMD64 + ARM64
├-- Embedding toolchain: ARM64
└─ Embedding vectorstore: ARM64
↓
Shared container-scan action
↓
Policy gate
↓
Publish + provenance

Source-code scan - black-duck-security-scan-ci.yml - non-container separate scan.

1. Pull requests receive a rapid source scan
2. Pushes to main and scheduled runs receive a full source scan.
3. Results use Black Duck project/version Arm:MCP / 1.0.
4. BLOCKER and CRITICAL violations fail the workflow.
5. SARIF is uploaded to GitHub Code Scanning, including after policy failures.

Shared container-scan action - All container workflows reuse blackduck-image-scan/action.yml.

1. Validates the image tag and expected architecture.
2. Checks the locally built image architecture.
3. Exports the image using docker image save.
4. Runs Black Duck with CONTAINER_SCAN & waits for Black Duck results.
5. Uploads SARIF to GitHub Code Scanning & retains it for 10 days
6. Deletes the temporary image archive.
7. Classifies failures: Exit code 8: policy violation & Other failure: scan or integration error.

MCP build-input images

Locked dependencies
↓
AMD64 local image ──→ Black Duck container scan
ARM64 local image ──→ Black Duck container scan
↓
Buildx publication with provenance
↓
Compare published configuration to scanned candidate
↓
Create multi-architecture index

Embedding toolchain

Build ARM64 toolchain locally
↓
Black Duck container scan
↓
Push the exact scanned local image
↓
Generate signed provenance attestation
↓
Propose toolchain digest update

Embedding vectorstore

Pinned toolchain + intrinsic chunks
↓
Generate embedding chunks
↓
Build scratch vectorstore image locally
↓
Black Duck container scan
↓
Push scanned image
↓
Attest and verify provenance
↓
Propose MCP embedding pin

Dry-run behavior - Manual workflows default to dry_run: true.

• Images are built and scanned.
• Black Duck results are uploaded.
• GHCR publication is skipped.
• Provenance and pin-update PRs are skipped.

[Copilot]

🟡 Changes recommended

The new composite action uploads SARIF via the third-party Black Duck action, which conflicts with the repo’s existing Scorecard-mitigated pattern of uploading SARIF via github/codeql-action/upload-sarif.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR introduces Black Duck policy gating (BLOCKER/CRITICAL) across both source-code scanning and container-image build pipelines, ensuring images are scanned and pass policy checks before being published to GHCR, with optional dry-run execution for manual dispatches.

Changes:

• Enables Black Duck policy gating for the source scan workflow and makes policy violations fail the job.
• Adds a shared composite action to scan locally built container images with Black Duck before publication and wires it into build-inputs, embedding-toolchain, and embedding-vectorstore workflows.
• Adds workflow_dispatch dry-run inputs and updates publication/provenance/attestation steps to run only when publishing is enabled.

File summaries

File	Description
mcp-local/tests/test_build_inputs.py	Updates workflow expectations to cover new provenance/attestation permissions and actions/attest usage.
.github/workflows/build-mcp-inputs.yml	Builds images locally, scans via shared Black Duck action before publish, adds dry-run gating and provenance on publication.
.github/workflows/build-embeddings.yml	Adds dry-run path with artifact handoff, scans vectorstore image before publish, and gates publication/attestation/promotion on publish-enabled runs.
.github/workflows/build-embedding-toolchain.yml	Builds locally, scans before publication, publishes the scanned image, and generates provenance attestation; adds dry-run gating.
.github/workflows/black-duck-security-scan-ci.yml	Enables policy failure severities and ensures policy violations fail the workflow; tightens SARIF upload condition.
.github/actions/blackduck-image-scan/action.yml	New composite action that validates image metadata, exports an archive, runs Black Duck container scan, uploads SARIF/artifacts, and classifies failures.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The composite action uploads SARIF unconditionally, which can fail when no SARIF file is produced for scan/integration errors and obscure the intended failure classification.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Lite

[github-advanced-security]

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

• The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
• Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
• You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

[brikin01]

Thanks for the PR! This looks good in terms of adding Black Duck scanning and release gates, with SBOM work to follow.

I left one comment about setting the Black Duck distribution type for each artifact, which should ideally reduce the number of license blockers we see. Hopefully adding that doesn't run into issues.

I also added one non-blocking note about the architecture-specific Docker Hub tags already tracked by STESOL-604. Maybe something I can tackle next week if deemed important.

[brikin01]

Could we set the Black Duck distribution type per image? Most of these images are internal build artifacts that aren’t distributed, while the final runtime image is public.

Black Duck defaults the project version distribution to EXTERNAL, and its license-risk calculation takes the distribution type into account.

Could we add a distribution input to this action and pass it through here?

--detect.project.version.distribution=${{ inputs.distribution }}
--detect.project.version.update=true

Based on Black Duck’s distribution definitions, the pipeline-only images could pass INTERNAL, while the public MCP runtime would likely pass OPENSOURCE because its source is publicly available.

I think update=true is needed because these Black Duck version names are reused; otherwise, the distribution is only set when a version is first created.

[brikin01]

Non-blocking: The production architecture tags are published before the Black Duck scan runs, so a failed or timed-out scan can still leave the public <version>-amd64 and <version>-arm64 tags on Docker Hub.

This issue is tracked by STESOL-604. We may want to consider addressing that sooner rather than later. Otherwise, this PR blocks the multi-architecture image and final release, but not publication of the architecture-specific tags.