Repository navigation

Build

Rename the fpv variant wfbng, after the wfb-ng it carries #1597

Workflow file for this run

	name: Build
	on:
	# There was no pull_request trigger here: a change landed unbuilt and the
	# nightly found out the next morning, or a user did. Building all 107 devices
	# per PR is what made that unaffordable, so .github/scripts/ci-matrix.py picks
	# only the devices a PR can reach. ready_for_review is not in the default set
	# (opened, synchronize, reopened), and the selector skips drafts, so without it
	# a PR opened as a draft would never build once marked ready.
	pull_request:
	branches:
	- master
	types: [opened, synchronize, reopened, ready_for_review]
	# There is deliberately no `schedule:` here any more, and adding one back
	# would undo a fix rather than restore a feature.
	#
	# This is the last stage of a three-stage release train — widgetii/majestic
	# publishes the majestic tarball, OpenIPC/firmware builds the images, this
	# builds from that tree. Each stage used to be its own GitHub cron guessing at
	# the previous stage's finish time (17:25, 22:30, 03:00 UTC). From 2026-08-26
	# GitHub's dispatcher began delivering scheduled events hours late and
	# independently per repo, and this repository was the worst affected of the
	# three: +10h38m and +11h49m on successive nights, because a 03:00 slot sits
	# in the deepest part of the congestion band. On the 08-28 cycle that made
	# this stage run FIRST, at 14:49, ahead of both the repositories it is
	# supposed to follow. Every run was green, because no stage checks that its
	# inputs are newer than the ones it used last time.
	#
	# The diagnostic, if this is ever doubted: on every late run
	# `created_at == run_started_at` to the second, so the delay was entirely
	# upstream of run creation, inside GitHub's dispatcher rather than in any
	# runner pool, concurrency group or matrix.
	#
	# The ordering now lives on the machine that runs majestic's self-hosted
	# runners, which dispatches each stage and waits for it to succeed before
	# starting the next. See widgetii/majestic docs/ci-runners.md.
	workflow_dispatch:
	inputs:
	platform:
	description: Platform
	required: false

	# Cancel a PR's earlier run when a new commit lands on it. Scheduled and
	# dispatched runs are deliberately left alone: they write releases, and a
	# cancelled publish leaves the nightly half-uploaded.
	concurrency:
	group: ${{ github.workflow }}-${{ github.event.pull_request.number \|\| github.ref }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	jobs:
	# Which devices can this change actually reach? Only ever narrows a
	# pull_request -- the nightly is decided by firmware HEAD, not by this repo's
	# diff, so every other event gets the whole matrix.
	#
	# The narrowing happens HERE, in a job that always runs, and never in a
	# `paths:` filter on the trigger: a filtered-out workflow does not run at all,
	# so a required check would stay pending forever and the PR could never merge.
	select:
	name: Select devices
	# Same guard as preflight below.
	if: >-
	github.repository == 'OpenIPC/builder' \|\|
	github.event_name == 'workflow_dispatch' \|\|
	(github.event_name == 'pull_request' && !github.event.repository.private)
	runs-on: ubuntu-latest
	outputs:
	matrix: ${{ steps.pick.outputs.matrix }}
	needs-build: ${{ steps.pick.outputs.needs-build }}
	steps:
	- uses: actions/checkout@v4
	- name: Check the selector still describes the tree
	run: python3 .github/scripts/ci-matrix.py --self-test
	- id: pick
	name: Pick the affected devices
	env:
	GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	run: python3 .github/scripts/ci-matrix.py

	preflight:
	name: Preflight
	# Guard rail for clones of this repo. A clone pushed to a new repository
	# (a private mirror, an internal re-host) inherits this file along with
	# and the full platform matrix then runs on that owner's Actions bill on
	# every internal PR they open. It used to also inherit a `cron:` and do
	# that nightly and unattended, publishing nightlies nobody reads; that half
	# went with the schedule trigger, the PR half did not. Mirrors
	# OpenIPC/firmware#2255.
	#
	# Unattended runs happen only on the canonical repo. Anywhere else a
	# workflow runs only if someone explicitly asked for it, or if it is both
	# free and attended -- which is what the pull_request clause covers, now
	# that this workflow has such a trigger.
	if: >-
	github.repository == 'OpenIPC/builder' \|\|
	github.event_name == 'workflow_dispatch' \|\|
	(github.event_name == 'pull_request' && !github.event.repository.private)
	runs-on: ubuntu-latest
	outputs:
	should_build: ${{ steps.gate.outputs.should_build }}
	head_sha: ${{ steps.gate.outputs.head_sha }}
	short_sha: ${{ steps.gate.outputs.short_sha }}
	build_id: ${{ steps.gate.outputs.build_id }}
	built_at: ${{ steps.gate.outputs.built_at }}
	steps:
	- uses: actions/checkout@v4
	- id: gate
	env:
	GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	run: \|
	HEAD=$(git rev-parse HEAD)
	SHORT=$(git rev-parse --short HEAD)
	BUILD_ID="nightly-$(date -u +%Y%m%d)-${SHORT}"
	BUILT_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
	# The nightly rebuilds unconditionally. builder.sh re-clones
	# OpenIPC/firmware at HEAD on every run, so the relevant inputs are
	# mostly outside this repo (firmware HEAD, toolchain release, kernel
	# tarball, vendor osdrv, individual package upstreams). The previous
	# gate skipped when only this repo's HEAD matched the last published
	# nightly, which made upstream fixes (e.g. an openhisilicon mipi_rx
	# regression fix that lands in firmware) invisible to users until
	# something here happened to change. Always build.
	#
	# Note this is why removing the `schedule:` trigger needed no gate
	# change here, unlike majestic and firmware: there is no event-name
	# condition to go stale, because there is no skip.
	echo "Build: $BUILD_ID (event=${{ github.event_name }})"
	echo "should_build=true" >> "$GITHUB_OUTPUT"
	echo "head_sha=$HEAD" >> "$GITHUB_OUTPUT"
	echo "short_sha=$SHORT" >> "$GITHUB_OUTPUT"
	echo "build_id=$BUILD_ID" >> "$GITHUB_OUTPUT"
	echo "built_at=$BUILT_AT" >> "$GITHUB_OUTPUT"

	# Prove the release credential BEFORE the matrix runs.
	#
	# OpenIPC/firmware run 33273333359 built 102 boards over two hours and then
	# died in `Publish releases` with "403 Resource not accessible by
	# integration". Nothing earlier in that run had any opinion about whether
	# it could publish, so the price of the misconfiguration was the whole
	# matrix. This repo builds 107 devices and publishes the same way.
	#
	# A draft release is not visible to anyone and creates no tag, so this
	# costs a couple of API calls and turns a long failure into a
	# ten-second one. Checking the secret is merely non-empty would not have
	# caught that incident at all: the old token existed and was simply
	# refused, so the probe has to attempt the write.
	#
	# Three details keep the probe from becoming a problem of its own:
	#
	# - The tag carries run_attempt as well as run_id. run_id is stable
	# across re-runs, so without the attempt a leftover draft from a
	# failed run would collide with its own re-run and report a valid
	# credential as broken -- a guard that fails wrongly is worse than no
	# guard, because it blocks good runs.
	# - Deletion happens from a trap, so an interrupted or cancelled job
	# still cleans up rather than leaving the draft behind.
	# - A best-effort sweep removes probe drafts stranded by earlier runs,
	# because cleanup.yml does not know this tag pattern. It is capped at
	# the 10 most recent releases and every failure is swallowed: listing
	# 100 releases here returns HTTP 504 (each carries hundreds of
	# assets), and a housekeeping step must never be what fails a build.
	#
	# Skipped on pull_request, which neither publishes nor has the secret.
	- name: Prove the release credential can write
	if: github.event_name != 'pull_request'
	env:
	GH_TOKEN: ${{ secrets.RELEASE_TOKEN }}
	GH_REPO: ${{ github.repository }}
	PROBE_TAG: ci-release-credential-probe-${{ github.run_id }}-${{ github.run_attempt }}
	run: \|
	set -euo pipefail

	if [ -z "${GH_TOKEN:-}" ]; then
	echo "::error::RELEASE_TOKEN is not set. The matrix would build"\
	"every device and then fail to publish. See the publish job."
	exit 1
	fi

	probe_id=""
	cleanup() {
	[ -n "$probe_id" ] \|\| return 0
	gh api -X DELETE "repos/${GH_REPO}/releases/${probe_id}" >/dev/null 2>&1 \|\| true
	}
	trap cleanup EXIT

	# Strand-clearing, entirely best effort.
	gh api "repos/${GH_REPO}/releases?per_page=10" \
	--jq '.[] \| select(.draft and (.tag_name \| startswith("ci-release-credential-probe-"))) \| .id' \
	2>/dev/null \| while read -r stale; do
	# gh writes its error body to STDOUT, so a failed list feeds this
	# loop lines of JSON rather than ids. Deleting them is harmless --
	# the calls just fail -- but logging "removing stranded probe
	# draft {"status": "401"}" is a confident lie in the log of a step
	# whose whole job is to be believed. Only act on real ids.
	case "$stale" in ''\|[!0-9]) continue ;; esac
	echo "removing stranded probe draft $stale"
	gh api -X DELETE "repos/${GH_REPO}/releases/${stale}" >/dev/null 2>&1 \|\| true
	done \|\| true

	# Retried, because a one-shot POST cannot tell a bad credential from
	# a bad minute. GitHub answers 403 for secondary rate limiting as
	# well as for refusal -- the publish step below retries six times for
	# exactly that reason -- and these repos return 504 on ordinary
	# release listings often enough to have been seen while writing this.
	# Reporting a blip as "the token is wrong" would send whoever reads
	# it to reissue a credential that was never the problem.
	#
	# Three attempts rather than the publish step's six: the value here is
	# failing fast, and 5s + 10s of backoff still absorbs a transient
	# while keeping a genuine misconfiguration well under a minute.
	attempt=0
	until probe_id=$(gh api -X POST "repos/${GH_REPO}/releases" \
	-f tag_name="$PROBE_TAG" -f name="$PROBE_TAG" \
	-F draft=true --jq .id); do
	attempt=$((attempt + 1))
	if [ "$attempt" -ge 3 ]; then
	echo "::error::RELEASE_TOKEN could not create a release in"\
	"${GH_REPO} after 3 attempts. If this is not a transient API"\
	"error, the token needs Contents: read and write."
	exit 1
	fi
	echo "attempt ${attempt} failed, retrying in $((attempt * 5))s"
	sleep $((attempt * 5))
	done
	echo "release credential OK"

	buildroot:
	name: Firmware
	needs: [preflight, select]
	if: >-
	needs.preflight.outputs.should_build == 'true' &&
	needs.select.outputs.needs-build == 'true'
	runs-on: ubuntu-latest
	env:
	BUILD_ID: ${{ needs.preflight.outputs.build_id }}
	BUILD_SHA: ${{ needs.preflight.outputs.head_sha }}
	BUILD_PLATFORM: ${{ matrix.platform }}

	strategy:
	fail-fast: false
	# The device list lives in .github/scripts/ci-matrix.py, which is
	# also what narrows it. Everything except pull_request gets all of it.
	matrix: ${{ fromJson(needs.select.outputs.matrix) }}
	steps:
	- name: Checkout source
	uses: actions/checkout@v4

	- name: Prepare firmware
	run: \|
	echo "nameserver 8.8.8.8" \| sudo tee /etc/resolv.conf
	echo CACHE_DATE=$(date +%m) >> ${GITHUB_ENV}
	INPUT=${{inputs.platform}}
	MATRIX=$(echo ${{matrix.platform}} \| cut -d_ -f1)
	if [ -z ${INPUT} ] \|\| [ ${INPUT} = ${MATRIX} ]; then
	echo RUN=${MATRIX} >> ${GITHUB_ENV}
	fi

	- name: Setup ccache
	if: env.RUN
	uses: actions/cache@v4
	with:
	path: /tmp/ccache
	key: ${{matrix.platform}}-${{env.CACHE_DATE}}

	# builder.sh does `rm -rf openipc` at the top of every run to force a
	# clean re-clone of OpenIPC/firmware, which also nukes the default
	# `openipc/output/dl/` directory Buildroot uses for downloaded source
	# tarballs. Without a cache, every nightly run re-downloads ~1-3 GB of
	# tarballs from upstream mirrors — and any one of them randomly
	# failing (e.g. cgit `snapshot/` URLs that regenerate tarballs per
	# request with shifting sha256, like wireguard-tools on
	# git.zx2c4.com) takes down the matrix entry for that board. See
	# kaeru wireguard-zx2c4-tarball-regeneration-hash-drift for the
	# specific incident this caching addresses.
	#
	# Cache to /tmp/builder-dl (outside the openipc/ tree so builder.sh's
	# rm -rf doesn't touch it) and point Buildroot at it via BR2_DL_DIR
	# env var (honored per Buildroot manual section "Location of source
	# tarballs"). Mirrors OpenIPC/firmware/.github/workflows/build.yml's
	# output/dl cache, adapted for builder's clean-clone behavior.
	- name: Setup dl cache
	if: env.RUN
	uses: actions/cache@v4
	with:
	path: /tmp/builder-dl
	key: builder-dl-${{env.CACHE_DATE}}
	restore-keys: builder-dl-

	- name: Refresh moving-ref package downloads
	if: env.RUN
	run: \|
	# builder caches BR2_DL_DIR (/tmp/builder-dl) under a monthly key
	# (date +%m) and actions/cache only saves on a key miss, so the first build
	# of the month freezes it for weeks. Packages pinned to a moving ref keep a
	# constant download filename, so buildroot reuses the stale tarball — even
	# though builder re-clones firmware HEAD each run for freshness. Two forms:
	# - dash: <pkg>-<ref>.tar.gz e.g. majestic-webui-dist.tar.gz
	# - dot: <pkg>.<ref>.tar.bz2 e.g. majestic.t31.lite.master.tar.bz2
	# The dot form is majestic's S3 tarball: it LOOKS S3-pinned but is a moving
	# `master` build re-uploaded every majestic CI run, with no .hash to force a
	# refetch — so it must be evicted too. (Empirically: a 06-14 nightly shipped a
	# ~2-week-old majestic-webui; later a t31 lite image shipped a pre-x-groups
	# majestic against the current webui — "No settings groups in schema".) Drop
	# both forms so buildroot re-fetches the current ref. Only genuinely
	# content-addressed tarballs (semver / SHA in the name) keep their cache.
	find /tmp/builder-dl -type f -regextype posix-extended \
	-regex '.*[-.](HEAD\|master\|main\|dist)\.tar\.(gz\|bz2\|xz)' \
	-print -delete 2>/dev/null \|\| true

	- name: Build firmware
	if: env.RUN
	run: \|
	export GIT_HASH=$(git rev-parse --short ${GITHUB_SHA})
	export GIT_BRANCH=${GITHUB_REF_NAME}
	echo GIT_HASH=${GIT_HASH} >> ${GITHUB_ENV}
	echo GIT_BRANCH=${GIT_BRANCH} >> ${GITHUB_ENV}

	mkdir -p /tmp/ccache
	ln -s /tmp/ccache ${HOME}/.ccache

	mkdir -p /tmp/builder-dl
	export BR2_DL_DIR=/tmp/builder-dl

	NAME=${{matrix.platform}}
	# Retry budget for transient upstream toolchain/CDN flakes
	# (matches OpenIPC/firmware/.github/workflows/build.yml).
	backoffs="30 60 120 300 600 1200"
	attempt=1
	for sleep_for in $backoffs ""; do
	bash builder.sh ${NAME} && break
	if [ -z "$sleep_for" ]; then
	echo "::error::build failed after ${attempt} attempts"
	exit 1
	fi
	echo "::warning::attempt ${attempt} failed, retrying after ${sleep_for}s"
	sleep "$sleep_for"
	attempt=$((attempt + 1))
	done
	cd openipc

	TIME=$(date -d @${SECONDS} +%M:%S)
	echo TIME=${TIME} >> ${GITHUB_ENV}
	COMMON=$(echo ${NAME} \| awk -F_ '{print NF-1}')

	NORFW=$(find output/images -name openipcnor)
	if [ ! -z ${NORFW} ]; then
	if [ ${COMMON} -eq 1 ]; then
	echo NORFW=openipc/${NORFW} >> ${GITHUB_ENV}
	else
	mv ${NORFW} ../${NAME}-nor.tgz
	echo NORFW=${NAME}-nor.tgz >> ${GITHUB_ENV}
	fi
	fi

	NANDFW=$(find output/images -name openipcnand)
	if [ ! -z ${NANDFW} ]; then
	if [ ${COMMON} -eq 1 ]; then
	echo NANDFW=openipc/${NANDFW} >> ${GITHUB_ENV}
	else
	mv ${NANDFW} ../${NAME}-nand.tgz
	echo NANDFW=${NAME}-nand.tgz >> ${GITHUB_ENV}
	fi
	fi

	if [ -z ${NORFW} ] && [ -z ${NANDFW} ]; then
	exit 1
	fi

	- name: Build size report
	if: env.NORFW \|\| env.NANDFW
	run: \|
	cd openipc
	NAME=${{matrix.platform}}
	make BOARD=${NAME} size-report \|\| \
	{ echo "::warning::size-report failed for ${NAME}"; exit 0; }
	SIZES=$(find output/images -name 'sizes.*.json' \| head -1)
	if [ -z "${SIZES}" ]; then
	exit 0
	fi
	# Mirror the .tgz rename trick: compound matrix entries
	# (`<soc>_<variant>_<vendor>-<model>`) share `<soc>_<variant>` and
	# would collide on the firmware-side `sizes.<soc>-<variant>.json`
	# filename when uploaded to the same release tag. Rename to
	# `sizes.<NAME>.json` so every compound device gets its own asset.
	COMMON=$(echo ${NAME} \| awk -F_ '{print NF-1}')
	if [ ${COMMON} -eq 1 ]; then
	echo "SIZES=openipc/${SIZES}" >> ${GITHUB_ENV}
	else
	mv ${SIZES} ../sizes.${NAME}.json
	echo "SIZES=sizes.${NAME}.json" >> ${GITHUB_ENV}
	fi

	# The Kconfig dependency graph and help text, which openipc.org's
	# firmware explorer needs for its "What if" tab -- and with it the
	# "Request this build" button. Mirrors OpenIPC/firmware's build.yml
	# (firmware#2171); da13271 brought the size report over a day before
	# that landed and this half was never followed, so no builder device
	# had the tab. Best-effort like the size report: an image never fails
	# for want of its graph.
	- name: Build kconfig graph
	if: env.NORFW \|\| env.NANDFW
	run: \|
	python3 -m pip install --user --break-system-packages kconfiglib \|\| \
	{ echo "::warning::kconfiglib install failed for ${{matrix.platform}}"; exit 0; }
	cd openipc
	NAME=${{matrix.platform}}
	make BOARD=${NAME} kconfig-graph \|\| \
	{ echo "::warning::kconfig-graph failed for ${NAME}"; exit 0; }
	# Same rename as the size report, for the same reason: compound
	# devices share the firmware-side `<soc>-<variant>` name, and
	# push_build.py takes the platform from the file name.
	COMMON=$(echo ${NAME} \| awk -F_ '{print NF-1}')
	KREPORTS=""
	for kind in kconfig-graph kconfig-help; do
	F=$(find output/images -name "${kind}.*.json" \| head -1)
	if [ -z "${F}" ]; then
	continue
	fi
	if [ ${COMMON} -eq 1 ]; then
	KREPORTS="${KREPORTS} openipc/${F}"
	else
	mv ${F} ../${kind}.${NAME}.json
	KREPORTS="${KREPORTS} ${kind}.${NAME}.json"
	fi
	done
	echo "KREPORTS=${KREPORTS}" >> ${GITHUB_ENV}

	# Stage everything this device produced into one flat directory, so the
	# artifact has a predictable shape no matter which of the two naming
	# schemes the build used (compound devices rename to <device>-nor.tgz in
	# the workspace root, simple ones leave openipc.<soc>-nor-<variant>.tgz
	# under openipc/output/images/).
	# Runs on pull_request too. Staging and the upload below are run-scoped
	# and touch no secret and no shared state, so unlike `Send binary` and the
	# `publish` job -- which stay gated, and must -- there is nothing here for
	# a PR to leak or clobber. Without this a contributor's PR compiles a full
	# image and then discards it, so they cannot test their own device profile
	# without asking a maintainer to dispatch build-one for them.
	- name: Stage artifacts
	if: env.NORFW \|\| env.NANDFW
	run: \|
	mkdir -p dist reports
	# The size report goes to its own artifact below, not with the image.
	if [ -n "${SIZES:-}" ] && [ -e "${SIZES}" ]; then
	cp -a "${SIZES}" reports/
	fi
	for f in ${KREPORTS:-}; do
	if [ -e "$f" ]; then
	cp -a "$f" reports/
	fi
	done
	for f in "${NORFW:-}" "${NANDFW:-}"; do
	# An `if` rather than a `&&` chain: the step runs under `bash -e`,
	# where a trailing `&&` that tests false is only harmless because
	# something after it resets the status.
	if [ -n "$f" ] && [ -e "$f" ]; then
	cp -a "$f" dist/
	fi
	done
	ls -la dist reports

	# Hand the images to the single `publish` job instead of writing releases
	# from here. 107 matrix jobs each deleting and re-uploading assets on the
	# SAME three shared releases is ~400 concurrent writes; OpenIPC/firmware
	# ran exactly that design and the releases API answered HTTP 500 under the
	# contention (run 27108181857: every board built, 14 jobs red in their
	# upload step). Consolidating every release write into one job removes the
	# concurrent-writer contention that causes it.
	#
	# `publish` carries its own `github.event_name != 'pull_request'`, so
	# producing fw-* on a PR cannot reach a release; it only makes the image
	# downloadable from the run that built it.
	#
	# Retention is per-event deliberately. A nightly's artifact is consumed by
	# `publish` inside the same run and no one downloads it afterwards, and a
	# nightly builds the entire device matrix -- holding those a week would
	# keep seven full matrices in storage instead of one, for no reader. A PR
	# builds a handful and has a person waiting on them, plausibly in another
	# timezone, so one day is too short a window there.
	- name: Upload build artifacts
	if: env.NORFW \|\| env.NANDFW
	uses: actions/upload-artifact@v4
	with:
	name: fw-${{ matrix.platform }}
	if-no-files-found: ignore
	retention-days: ${{ github.event_name == 'pull_request' && 7 \|\| 1 }}
	path: dist/*

	# The size report and kconfig graph, for the `report` job that pushes
	# them to openipc.org.
	# A few tens of KB, kept a week -- unlike the image, which publish
	# consumes within the run -- so a push that could not reach openipc.org
	# can be re-run days later. Optional, so is this.
	- name: Upload report
	if: github.event_name != 'pull_request' && (env.NORFW \|\| env.NANDFW)
	uses: actions/upload-artifact@v4
	continue-on-error: true
	with:
	name: reports-${{ matrix.platform }}
	if-no-files-found: ignore
	retention-days: 7
	path: reports/*

	- name: Send binary
	if: github.event_name != 'pull_request' && env.NORFW
	run: \|
	TG_MSG="Build: ${BUILD_ID}\nCommit: ${GIT_HASH}\nBranch: ${GIT_BRANCH}\nTime: ${TIME}\n\n"
	TG_ICON="\xE2\x9C\x85 GitHub Actions"
	TG_HEADER=$(echo -e ${TG_MSG}${TG_ICON})
	TG_TOKEN=${{secrets.TELEGRAM_TOKEN_BOT_OPENIPC}}
	TG_CHANNEL=${{secrets.TELEGRAM_CHANNEL_OPENIPC_DEV}}
	HTTP=$(curl -s -o /dev/null -w %{http_code} https://api.telegram.org/bot${TG_TOKEN}/sendDocument -F chat_id=${TG_CHANNEL} -F caption="${TG_HEADER}" -F document=@${NORFW})
	echo Telegram response: ${HTTP}

	# All release writes happen here, once, so only a SINGLE job ever touches the
	# shared `nightly`/`latest` releases and the per-run dated one. Best-effort by
	# design: publish whatever the matrix produced even if some devices failed,
	# which matches enrich_manifest.py -- it indexes whatever assets exist. Never
	# runs on a pull request; there are no artifacts to publish there.
	publish:
	name: Publish releases
	needs: [preflight, select, buildroot]
	# The leading status function is load-bearing. A job whose `needs` failed is
	# skipped by default, and an `if:` lifts that only when it CONTAINS a status
	# function -- reading `needs.buildroot.result` inside the `contains()` below
	# does not count. So since #121 gave this job the matrix as a dependency,
	# any single failing device skipped it outright and the whole release went
	# unwritten: 2026-08-24 lost every device's nightly to one failure out of 98
	# (ssc333_lite_meari-speed-6s), and 2026-08-21 lost it to two out of 98.
	# There is no nightly-20260821-* or nightly-20260824-* release as a result.
	# Before #121 each device uploaded from its own matrix job, so a bad device
	# took down only itself; this restores that without giving up the
	# single-writer fix that stopped the concurrent-upload rate limits.
	#
	# `!cancelled()` rather than `always()`: the `contains()` already drops a
	# cancelled or skipped matrix, but a run cancelled AFTER a green matrix
	# would still start uploading under `always()`, and a half-written nightly
	# is the thing this job exists to avoid.
	if: >-
	!cancelled() &&
	github.event_name != 'pull_request' &&
	needs.preflight.outputs.should_build == 'true' &&
	contains(fromJSON('["success", "failure"]'), needs.buildroot.result)
	runs-on: ubuntu-latest
	permissions:
	contents: write
	steps:
	# continue-on-error only so that a matrix where every device failed --
	# and therefore uploaded no fw-* artifact at all -- does not hard-fail
	# here before the gate can report why. Collect below turns that back into
	# a failure for every case except that one, so a download that breaks can
	# no longer publish nothing and call it success.
	- name: Download build artifacts
	id: download
	uses: actions/download-artifact@v4
	continue-on-error: true
	with:
	pattern: fw-*
	path: dist
	merge-multiple: true

	- name: Collect assets
	id: collect
	run: \|
	mkdir -p dist
	# tr because wc pads its output on some platforms (BSD does, the GNU
	# coreutils on the runner does not) and a padded " 0" would read
	# as non-empty to the `!= '0'` guard below.
	count=$(find dist -type f \| wc -l \| tr -d '[:space:]')
	echo "Collected ${count} asset(s):"
	ls -la dist \|\| true
	echo "count=${count}" >> "$GITHUB_OUTPUT"

	# A matrix that went green must have produced artifacts. Zero assets
	# after that means the download failed, not that there was nothing to
	# publish -- and publishing nothing quietly is exactly how a nightly
	# goes missing while the run stays green.
	if [ "${{ needs.buildroot.result }}" = "success" ]; then
	if [ "${{ steps.download.outcome }}" != "success" ]; then
	echo "::error::artifact download failed after a fully successful matrix"
	exit 1
	fi
	if [ "${count}" -eq 0 ]; then
	echo "::error::no artifacts collected although every device built"
	exit 1
	fi
	elif [ "${count}" -eq 0 ]; then
	echo "::notice::matrix failed and produced nothing; no release to write"
	else
	# The partial case this job was always meant to serve, unreachable
	# until the `!cancelled()` above. Say so out loud: the devices that
	# did not build keep their PREVIOUS image on nightly/latest, which
	# is the right outcome for a flasher but is invisible next to the
	# freshly-written ones.
	echo "::warning::partial publish: matrix failed, writing the ${count} asset(s) that did build; nightly and latest keep their previous image for every device that did not build — see the failed jobs in this run for which"
	fi

	# Drive the release writes with gh rather than softprops/action-gh-release,
	# which uploads assets CONCURRENTLY with no throttle or retry knob. Firing
	# a few hundred asset writes at three releases trips GitHub's per-actor
	# secondary rate limit; here every asset goes up one at a time, paced under
	# the per-minute mutation ceiling, with exponential backoff so a transient
	# 403/429 is retried rather than failing the nightly.
	- name: Publish releases (paced, retry-aware)
	if: steps.collect.outputs.count != '0'
	env:
	# Not GITHUB_TOKEN. The default token's release-write depends on
	# who triggered the run: a `schedule:` run executes as the repository's
	# scheduling identity and could write releases, but a dispatched run
	# executes as whoever dispatched it, and for an app/OAuth-backed
	# identity the API answers "403 Resource not accessible by
	# integration" even though the job declares `contents: write` and the
	# run log confirms the token was granted it. OpenIPC/firmware hit
	# exactly this on 2026-08-29 (its run 33273333359 built for two hours
	# and then could not publish). This repo publishes the same way and
	# was next in line. An explicit credential also makes publishing
	# independent of who or what starts the run.
	GH_TOKEN: ${{ secrets.RELEASE_TOKEN }}
	GH_REPO: ${{ github.repository }}
	BUILD_ID: ${{ needs.preflight.outputs.build_id }}
	HEAD_SHA: ${{ needs.preflight.outputs.head_sha }}
	SHORT_SHA: ${{ needs.preflight.outputs.short_sha }}
	BUILT_AT: ${{ needs.preflight.outputs.built_at }}
	run: \|
	set -euo pipefail

	gh_retry() {
	local n=0 max=6 delay=15
	until "$@"; do
	n=$((n + 1))
	if [ "$n" -ge "$max" ]; then
	echo "::error::gh failed after ${max} attempts: $*"
	return 1
	fi
	echo "::warning::gh attempt ${n} failed, sleeping ${delay}s before retry"
	sleep "$delay"
	delay=$((delay * 2))
	done
	}

	ensure_release() { # <tag> [extra gh release create args...]
	local tag="$1"; shift
	if ! gh release view "$tag" >/dev/null 2>&1; then
	gh_retry gh release create "$tag" --target "$HEAD_SHA" --notes "$NOTES" "$@"
	fi
	}

	# One asset at a time, paced ~1/s to stay under the per-minute
	# content-mutation ceiling that triggers the secondary rate limit.
	upload_paced() { # <tag> <file>...
	local tag="$1"; shift
	local total=$# i=0
	for f in "$@"; do
	i=$((i + 1))
	echo "[${tag}] (${i}/${total}) ${f##*/}"
	gh_retry gh release upload "$tag" "$f" --clobber
	sleep 1
	done
	}

	NOTES=$(printf 'sha=%s\nshort=%s\nbuilt_at=%s\n' "$HEAD_SHA" "$SHORT_SHA" "$BUILT_AT")

	# Releases carry firmware and nothing else. The per-device size
	# reports that used to ride along as sizes.*.json sidecars reach
	# openipc.org once, from the report job below, out of their own
	# reports-* artifacts.
	mapfile -t IMAGES < <(find dist -maxdepth 1 -type f -name '*.tgz' \| sort)

	# Nothing moves when there is no firmware to move it to. A dated
	# release with nothing in it, or the two tags users flash from
	# pointing at one, is worse than leaving everything on yesterday's.
	if [ "${#IMAGES[@]}" -eq 0 ]; then
	echo "::warning::no images in this build; no dated release, and nightly and latest stay where they are"
	else
	# --- dated: immutable per-build history ---
	ensure_release "$BUILD_ID" --prerelease --title "$BUILD_ID"
	upload_paced "$BUILD_ID" "${IMAGES[@]}"

	# --- rolling nightly: move tag + body to this build, refresh images ---
	ensure_release nightly
	gh_retry gh release edit nightly --notes "$NOTES"
	gh_retry gh api -X PATCH "repos/${GH_REPO}/git/refs/tags/nightly" -f sha="$HEAD_SHA" -F force=true
	upload_paced nightly "${IMAGES[@]}"

	# --- latest: legacy alias ---
	ensure_release latest
	gh_retry gh api -X PATCH "repos/${GH_REPO}/git/refs/tags/latest" -f sha="$HEAD_SHA" -F force=true
	upload_paced latest "${IMAGES[@]}"

	# --- latest: the wfbng images under their fpv names ---
	# The variant was fpv until it was renamed wfbng, after the wfb-ng
	# it carries. Cameras flashed before then still fetch the old
	# names from latest: the generic ones by BUILD_OPTION, and the
	# devices by the `upgrade` URL their customizer.sh wrote into the
	# bootloader environment, which an upgrade keeps (S30customizer
	# runs once). So latest keeps answering both, with no end date: a
	# device camera would otherwise lose its updates silently.
	ALIASES=()
	mkdir -p dist-fpv
	for f in "${IMAGES[@]}"; do
	b=${f##*/}
	case "$b" in
	openipc.*-wfbng.tgz) a=${b%-wfbng.tgz}-fpv.tgz ;;
	_wfbng_) a=${b/_wfbng_/_fpv_} ;;
	*) continue ;;
	esac
	cp -- "$f" "dist-fpv/$a"
	ALIASES+=("dist-fpv/$a")
	done
	if [ "${#ALIASES[@]}" -gt 0 ]; then
	upload_paced latest "${ALIASES[@]}"
	fi
	fi

	# openipc.org learns about the build here, once: which device images were
	# published and their digests, and each device's size report. It never polls
	# GitHub for any of it, and none of it is uploaded to a release. The
	# contract, and who may push, is
	# https://github.com/OpenIPC/website/blob/master/service/internal/builds/PUSH.md
	#
	# Authorization is this job's OIDC token (audience https://openipc.org),
	# which openipc.org checks against the repository, this workflow and
	# master. There is no secret to store or rotate.
	#
	# A job of its own, after publish and outside ci-gate, so that openipc.org
	# being down costs a published nightly nothing, and the push can be re-run by
	# itself: it is idempotent by build id. It reads the release and the
	# reports-* artifacts, which are kept a week.
	report:
	name: Report build to openipc.org
	needs: [preflight, publish]
	if: >-
	!cancelled() &&
	github.event_name != 'pull_request' &&
	needs.publish.result == 'success'
	runs-on: ubuntu-latest
	# id-token is what authorizes the push, so nothing but this repository's
	# own push script, at the built commit, runs here: the artifacts are read
	# as data, never executed. actions: read is for counting the reports-*
	# artifacts the matrix uploaded (the guard below).
	permissions:
	id-token: write
	contents: read
	actions: read
	timeout-minutes: 20
	steps:
	# First: checkout cleans the workspace, so anything downloaded before it
	# would be deleted.
	- uses: actions/checkout@v4
	with:
	ref: ${{ needs.preflight.outputs.head_sha }}
	sparse-checkout: .github/scripts

	# The reports only, outside the checkout. The images are described from
	# the release itself, so nothing here depends on the one-day fw-*
	# artifacts.
	- name: Download reports
	id: reports
	uses: actions/download-artifact@v4
	continue-on-error: true
	with:
	pattern: reports-*
	path: ${{ runner.temp }}/reports
	merge-multiple: true

	# What was published, as GitHub has it: each device image's size and the
	# sha256 GitHub computed on upload. publish creates no dated release when
	# there were no images, and then there is nothing to report.
	#
	# "No reports produced" and "reports lost" look the same to the push, and
	# only the first is acceptable: if the matrix uploaded any reports-*
	# artifact, some report has to be here.
	- name: Push the build
	env:
	GH_TOKEN: ${{ github.token }}
	GH_REPO: ${{ github.repository }}
	RUN_ID: ${{ github.run_id }}
	REPORTS: ${{ runner.temp }}/reports
	DOWNLOAD: ${{ steps.reports.outcome }}
	BUILD_ID: ${{ needs.preflight.outputs.build_id }}
	HEAD_SHA: ${{ needs.preflight.outputs.head_sha }}
	BUILT_AT: ${{ needs.preflight.outputs.built_at }}
	run: \|
	set -euo pipefail
	if ! gh release view "$BUILD_ID" --json assets > release.json; then
	echo "::notice::no release $BUILD_ID; nothing to report"
	exit 0
	fi
	uploaded=$(gh api "repos/${GH_REPO}/actions/runs/${RUN_ID}/artifacts" --paginate \
	--jq '.artifacts[] \| select(.name \| startswith("reports-")) \| select(.expired \| not) \| .name' \| wc -l)
	# A missing directory is zero reports, not a failed step.
	found=0
	if [ -d "$REPORTS" ]; then
	found=$(find "$REPORTS" -maxdepth 1 -name '*.json' \| wc -l)
	fi
	echo "reports-* artifacts uploaded: ${uploaded}; report files here: ${found}"
	if [ "$uploaded" -gt 0 ] && { [ "$DOWNLOAD" != success ] \|\| [ "$found" -eq 0 ]; }; then
	echo "::error::the matrix uploaded ${uploaded} reports-* artifact(s) but none reached this job (download: ${DOWNLOAD}); refusing to push devices without their reports"
	exit 1
	fi
	python3 .github/scripts/push_build.py \
	--source builder \
	--build-id "$BUILD_ID" \
	--sha "$HEAD_SHA" \
	--built-at "$BUILT_AT" \
	--reports "$REPORTS" \
	--release-json release.json \
	--match '\.tgz$'

	# Single umbrella status check covering the whole run, so branch protection
	# can require one context instead of a hardcoded "Build (<device>)" per
	# device -- the matrix is now dynamic, and its job names change per PR.
	# There was no gate here at all before: nothing reported a verdict on a
	# change, because nothing built one.
	ci-gate:
	name: CI Gate
	needs: [select, preflight, buildroot, publish]
	# always() so the gate still reports when a needed job fails -- but not on
	# a run the guard above deliberately skipped, where it would read
	# select=skipped and turn a saved build into a red run.
	if: >-
	always() && (github.repository == 'OpenIPC/builder' \|\|
	github.event_name == 'workflow_dispatch' \|\|
	(github.event_name == 'pull_request' && !github.event.repository.private))
	runs-on: ubuntu-latest
	steps:
	- name: Require selection + device matrix to succeed
	run: \|
	echo "select=${{ needs.select.result }} preflight=${{ needs.preflight.result }} buildroot=${{ needs.buildroot.result }} publish=${{ needs.publish.result }}"
	if [ "${{ needs.select.result }}" != "success" ]; then
	echo "::error::device selection did not succeed"; exit 1
	fi
	if [ "${{ needs.preflight.result }}" != "success" ]; then
	echo "::error::preflight did not succeed"; exit 1
	fi
	# Check the matrix against what select asked for. Blanket-accepting
	# 'skipped' would mean a selector bug that narrowed a real change
	# down to nothing reads as a green PR.
	if [ "${{ needs.select.outputs.needs-build }}" = "true" ] && \
	[ "${{ needs.preflight.outputs.should_build }}" = "true" ]; then
	if [ "${{ needs.buildroot.result }}" != "success" ]; then
	echo "::error::device matrix result=${{ needs.buildroot.result }}"; exit 1
	fi
	echo "device matrix OK (success)"
	else
	# Nothing in the change reaches a build, or the PR is still a draft.
	if [ "${{ needs.buildroot.result }}" != "skipped" ]; then
	echo "::error::device matrix ran when it should not have (${{ needs.buildroot.result }})"; exit 1
	fi
	echo "device matrix OK (no devices to build)"
	fi
	# publish is skipped on PRs and on a run that built nothing; only a
	# real failure of the single release-writer should fail the gate.
	case "${{ needs.publish.result }}" in
	success\|skipped) echo "publish OK (${{ needs.publish.result }})" ;;
	*) echo "::error::publish result=${{ needs.publish.result }}"; exit 1 ;;
	esac

Navigation Menu

Repository navigation

Rename the fpv variant wfbng, after the wfb-ng it carries #1597

Workflow file

Rename the fpv variant wfbng, after the wfb-ng it carries #1597

Uh oh!

Workflow file for this run