Skip to content

Firefly-2099: Harden and improve Firefly standalone install #2013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
robyww wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Firefly-2099: Harden and improve Firefly standalone install #2013

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 30 additions & 7 deletions .github/workflows/build_publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ on:
workflow_dispatch:
inputs:
git_tag:
description: "Git tag to use(e.g. 'release-xxxx.x.x' or a full git tag)"
description: "Git tag or branch to use (e.g. 'release-xxxx.x.x' or a full git tag)"
required: true
type: string
img_tag:
description: "Optional: Docker image tag(e.g. 'xxxx.x.x'); If omitted, `git_tag` will be used."
description: "Optional: Docker image tag (e.g. 'xxxx.x.x'); If omitted, `git_tag` will be used."
required: false
type: string
push_image:
Expand All @@ -20,6 +20,11 @@ on:
description: "Optional: Firefly release tag for standalone.zip artifact (e.g. release-xxxx.x.x)"
required: false
type: string
standalone_only:
description: "standalone.zip only (no docker push, no GitHub release)"
required: false
default: false
type: boolean
release:
types: [published]

Expand Down Expand Up @@ -90,7 +95,7 @@ jobs:
# Login to GHCR (only if pushing)
# ------------------------------------------------------------
- name: Login to GHCR
if: github.event_name == 'release' || inputs.push_image
if: (github.event_name == 'release' || inputs.push_image) && !inputs.standalone_only
uses: docker/login-action@v3
with:
registry: ghcr.io
Expand All @@ -106,7 +111,7 @@ jobs:
context: .
file: firefly/docker/Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name == 'release' || inputs.push_image }}
push: ${{ (github.event_name == 'release' || inputs.push_image) && !inputs.standalone_only}}
# docker buildx does not allow uppercase letters in tags, so we convert to lowercase here
tags: ghcr.io/caltech-ipac/firefly:${{ steps.resolve_tags.outputs.tag }}
build-args: |
Expand All @@ -120,7 +125,7 @@ jobs:
# `standalone-builder` layer (gradle standalone:zip) runs.
# ------------------------------------------------------------
- name: Build and export standalone.zip
if: github.event_name == 'release' || inputs.release_tag != ''
if: github.event_name == 'release' || inputs.release_tag != '' || inputs.standalone_only
uses: docker/build-push-action@v6
with:
context: .
Expand All @@ -140,21 +145,39 @@ jobs:
# using `gh release upload` requires setting `contents: write`
# ------------------------------------------------------------
- name: Add standalone.zip to release artifacts
if: github.event_name == 'release' || inputs.release_tag != ''
if: (github.event_name == 'release' || inputs.release_tag != '') && !inputs.standalone_only
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="${{ github.event_name == 'release' && github.event.release.tag_name || inputs.release_tag }}"
gh release upload "$TAG" --repo "${{ github.repository }}" ./dist-zip/standalone.zip --clobber

# ------------------------------------------------------------
# standalone_only: publish standalone.zip as a workflow
# artifact instead of a release asset. Ephemeral, private to the run.
# ------------------------------------------------------------
- name: Upload standalone.zip as workflow artifact
id: upload_standalone_artifact
if: inputs.standalone_only
uses: actions/upload-artifact@v4
with:
name: standalone-zip
path: ./dist-zip/standalone.zip

- name: Print artifact URL
if: inputs.standalone_only
run: |
echo "Artifact URL: ${{ steps.upload_standalone_artifact.outputs.artifact-url }}" >> $GITHUB_STEP_SUMMARY
echo "nightly.link (no login required): https://nightly.link/${{ github.repository }}/actions/runs/${{ github.run_id }}/standalone-zip.zip" >> $GITHUB_STEP_SUMMARY

# ------------------------------------------------------------
# Package and push Helm chart to GHCR
# ------------------------------------------------------------
- name: Set up Helm
uses: azure/setup-helm@v4

- name: Publish Helm chart
if: github.event_name == 'release' || inputs.push_image
if: (github.event_name == 'release' || inputs.push_image) && !inputs.standalone_only
run: |
if [[ ! -f firefly/helm/Chart.yaml ]]; then
echo "No Helm chart found, skipping."
Expand Down
88 changes: 80 additions & 8 deletions bin/get-firefly
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,81 @@
#!/bin/bash
INSTALL_SCRIPT="https://raw.githubusercontent.com/Caltech-IPAC/firefly/refs/heads/dev/bin/install.sh"
# use the following line for testing the code from a PR, modify for your PR
# INSTALL_SCRIPT="https://raw.githubusercontent.com/Caltech-IPAC/firefly/refs/heads/FIREFLY-1980-standalone/bin/install.sh"

curl -s ${INSTALL_SCRIPT} > ./install.sh
chmod +x ./install.sh
./install.sh -dontConfirm "$@"
/bin/rm -f ./install.sh

# --------------------------
# This script start the Firefly install on an end-user machine
# --------------------------

# --------------------------
# How to use
#
# Production:
# curl -L https://raw.githubusercontent.com/Caltech-IPAC/firefly/refs/heads/dev/bin/get-firefly | bash
#
# Testing in a branch:
# curl -L https://raw.githubusercontent.com/Caltech-IPAC/firefly/refs/heads/FIREFLY-2099-stand-hard/bin/get-firefly | FIREFLY_BRANCH=Firefly-xxx bash
# --------------------------

ref="${FIREFLY_BRANCH:-dev}"
INSTALL_SCRIPT="https://raw.githubusercontent.com/Caltech-IPAC/firefly/refs/heads/${ref}/bin/install.sh"

downloadedInstallScript=""
jobArtifactZip=""
jobExtractDir=""
trap '[ -n "$downloadedInstallScript" ] && rm -f "$downloadedInstallScript"; [ -n "$jobArtifactZip" ] && rm -f "$jobArtifactZip"; [ -n "$jobExtractDir" ] && rm -rf "$jobExtractDir"' EXIT

downloadedInstallScript=$(mktemp -t firefly-install.XXXXXX) || { echo "Failed to create a temporary file"; exit 1; }

curl -fsSL "${INSTALL_SCRIPT}" -o "$downloadedInstallScript"
curlStatus=$?
if [ $curlStatus -ne 0 ] || [ ! -s "$downloadedInstallScript" ]; then
echo "Failed to download the Firefly installer from $INSTALL_SCRIPT"
exit 1
fi
chmod +x "$downloadedInstallScript"

# --------------------------
# The following two environment variables are only for testing.

# FIREFLY_RUN_ID: when set, install the build of standalone.zip from that
# specific GitHub Actions run (via nightly.link) instead of the latest formal GitHub
# release. If testing is set up correctly then FIREFLY_RUN_ID should not be necessary
#
# FIREFLY_BRANCH: if FIREFLY_RUN_ID isn't set but this is, auto-resolve the latest
# successful workflow_dispatch run on that branch.
# Note this can't tell whether that run was actually a standalone_only
# build - if not, nightly.link will just report no artifact found for it.
#
# --------------------------

installUrlArgs=()
if [ -z "$FIREFLY_RUN_ID" ] && [ -n "$FIREFLY_BRANCH" ]; then
if ! command -v jq > /dev/null 2>&1; then
echo "jq is required to resolve FIREFLY_BRANCH to a run id; install jq or set FIREFLY_RUN_ID directly"
exit 1
fi
runApiUrl="https://api.github.com/repos/Caltech-IPAC/firefly/actions/workflows/build_publish.yml/runs?branch=${FIREFLY_BRANCH}&status=success&event=workflow_dispatch"
FIREFLY_RUN_ID=$(curl -fsSL "$runApiUrl" | jq -r '.workflow_runs[0].id // empty')
if [ -z "$FIREFLY_RUN_ID" ]; then
echo "No successful workflow_dispatch run found for branch $FIREFLY_BRANCH"
exit 1
fi
fi

if [ -n "$FIREFLY_RUN_ID" ]; then
runLinkUrl="https://nightly.link/Caltech-IPAC/firefly/actions/runs/${FIREFLY_RUN_ID}/standalone-zip.zip"
jobArtifactZip=$(mktemp -t firefly-build-artifact.XXXXXX) || { echo "Failed to create a temporary file"; exit 1; }
curl -fsSL "$runLinkUrl" -o "$jobArtifactZip"
curlStatus=$?
if [ $curlStatus -ne 0 ] || [ ! -s "$jobArtifactZip" ]; then
echo "Failed to download the build from $runLinkUrl"
exit 1
fi
jobExtractDir=$(mktemp -d -t firefly-build-extract.XXXXXX) || { echo "Failed to create a temporary directory"; exit 1; }
if ! unzip -oq "$jobArtifactZip" -d "$jobExtractDir" || [ ! -f "$jobExtractDir/standalone.zip" ]; then
echo "Failed to expand the build artifact from $runLinkUrl"
exit 1
fi
installUrlArgs=(-url "$jobExtractDir/standalone.zip")
fi

# now execute the real install script
"$downloadedInstallScript" -dontConfirm "${installUrlArgs[@]}" "$@"
Loading