Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 43 additions & 110 deletions base/comps/telegraf/generate_source_tarball.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
#
# Generates a reproducible vendored Go-module tarball for telegraf.
# Vendors Go dependencies for telegraf and produces a vendor-only .tar.gz archive.
#
# NOTE: This is an out-of-band maintainer tool — it is NEVER invoked during rpmbuild.
# It is run by hand (or by the sources-upload pipeline) to (re)produce the vendor
# archive, which is then uploaded to the lookaside and pinned by hash as
# `Source1: %{archivename}-vendor.tar.bz2`. The spec's %build consumes that prebuilt
# archive; it never runs `go mod vendor`. Consequently the pinned timestamp here only
# affects reproducibility of this generation step, not the package build.
# It is run by hand (or by the sources-upload pipeline) to produce the vendor archive
# (`%{archivename}-vendor.tar.gz`), which is then uploaded to the lookaside and pinned
# by hash in telegraf.comp.toml. The spec's %build consumes that prebuilt archive;
# it never runs `go mod vendor`. azldev packages the generated `vendor/` directory into
# the custom source archive, whose pinned hash is verified before the package build.
#
# The custom (minimal-plugin) build still vendors the *full* dependency tree so
# the build is hermetic and offline; only the selected plugins are compiled in
Expand All @@ -19,39 +19,23 @@
# ---------------------------------------------------------------------------
# Reproducibility contract
# ---------------------------------------------------------------------------
# For supply-chain reasons this vendor archive must be reproducible (within
# toolchain tolerance). Every input that determines the archive's contents is
# HARD-CODED below, so this script alone documents exactly how the pinned
# vendor tarball was produced — there are no version/output arguments to get
# wrong or forget to record.
# For supply-chain traceability, every input that determines the vendored source
# tree is HARD-CODED below. This script documents exactly how it was produced —
# there are no version/output arguments to get wrong or forget to record.
#
# By default the script downloads the upstream source tarball itself from the
# hard-coded SRC_TARBALL_URI, so it controls the entire source path — the package
# version (hard-coded below) is the only thing that varies. An optional path
# argument overrides the download with a pre-fetched local copy (e.g. an
# air-gapped or lookaside mirror). In both cases the tarball's SHA512 is verified
# against the hard-coded value before vendoring. This is not a security mechanism
# (the script can be edited); it exists to stop us from accidentally vendoring the
# wrong input.
# The script expects the upstream source tarball to already be present in the current
# working directory. Its SHA512 is verified against the hard-coded value
# before vendoring. This is not a security mechanism (the script can be edited); it
# exists to stop us from accidentally vendoring the wrong input.
#
# The produced archive's SHA512 is compared against the hard-coded expected
# value at the end. A mismatch is a non-fatal warning: it almost always means a
# toolchain difference (the Go toolchain writes vendor/modules.txt, so the Go
# version dominates), which is the "certain tolerance" for reproducibility. To
# reproduce the pinned hash byte-for-byte, use the canonical toolchain below.
# Canonical toolchain used to vendor dependencies:
# go : go1.26.4
#
# Canonical toolchain used to produce EXPECTED_VENDOR_SHA512:
# go : go1.26.4
# tar : tar (GNU tar) 1.35
# bzip2 : bzip2 1.0.8
#
# Bumping the telegraf version: update PKG_VERSION, SRC_TARBALL_URI,
# SRC_TARBALL_SHA512, and (after a canonical-toolchain run) EXPECTED_VENDOR_SHA512
# below, then update the matching hashes in telegraf.comp.toml.
# Bumping the telegraf version: update PKG_VERSION and SRC_TARBALL_SHA512 below,
# then update the matching hashes in telegraf.comp.toml.
#
# Usage:
# ./generate_source_tarball.sh # download the pinned tarball
# ./generate_source_tarball.sh <path-to-tarball> # use a pre-fetched copy
# ./generate_source_tarball.sh

set -euo pipefail

Expand All @@ -61,43 +45,27 @@ set -euo pipefail
readonly PKG_NAME="telegraf"
readonly PKG_VERSION="1.38.2"

# Upstream source archive (the one argument must match this):
# Upstream source archive — expected in the current working directory.
readonly SRC_TARBALL_NAME="telegraf-${PKG_VERSION}.tar.gz"
readonly SRC_TARBALL_URI="https://github.com/influxdata/telegraf/archive/v${PKG_VERSION}/${SRC_TARBALL_NAME}"
readonly SRC_TARBALL_SHA512="36c419978e98da9809e18865053399dd2198abc3d650f54424e3eb359ff8dfcb615f8b0a82dba484d03acfe5abe51a160a41ef486e5602f99400bd69e7afe48d"

# Expected output (verified, non-fatally, at the end). Reproducible with the
# canonical toolchain documented in the header; may differ otherwise.
readonly EXPECTED_VENDOR_SHA512="1108fe48086a7051c5cb89935c6de1c675c3ea8212a979d147ad0c03aef327c6234fa9eee292e4f9594ba9ec2cb757fc9eff46630aea43551bca3d948b30b27f"

# Output lands next to this script (not committed; upload to the lookaside).
readonly OUT_FOLDER="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# azldev's custom source workflow archives the `vendor/` directory placed here.
readonly OUT_FOLDER="/azldev-gen/output"

usage() {
echo "Usage: ${0##*/} [path-to-${SRC_TARBALL_NAME}]" >&2
echo "Usage: ${0##*/}" >&2
echo >&2
echo "Produces ${PKG_NAME}-${PKG_VERSION}-vendor.tar.bz2 (reproducible)." >&2
echo "With no argument the pinned source tarball is downloaded from:" >&2
echo " ${SRC_TARBALL_URI}" >&2
echo "Pass a path to use a pre-fetched local copy instead (offline override)." >&2
echo "Expects ${SRC_TARBALL_NAME} in the current working directory." >&2
echo "Produces ${OUT_FOLDER}/vendor for azldev to archive as" >&2
echo "${PKG_NAME}-${PKG_VERSION}-vendor.tar.gz." >&2
}

# ---------------------------------------------------------------------------
# Argument: an OPTIONAL path to a pre-fetched source tarball. With no argument
# the script downloads the tarball itself from the pinned SRC_TARBALL_URI, so it
# controls the entire source path; only the (hard-coded) package version varies.
# ---------------------------------------------------------------------------
if [ "$#" -gt 1 ]; then
usage
exit 1
if [ "$#" -gt 0 ]; then
case "${1}" in
-h|--help) usage; exit 0 ;;
*) echo "Error: unexpected argument '${1}'" >&2; usage; exit 1 ;;
esac
fi
case "${1:-}" in
-h|--help)
usage
exit 0
;;
esac
SRC_OVERRIDE="${1:-}"

echo "Package -> ${PKG_NAME}-${PKG_VERSION}"
echo "Output folder -> ${OUT_FOLDER}"
Expand All @@ -111,21 +79,15 @@ function cleanup {
trap cleanup EXIT

# ---------------------------------------------------------------------------
# Obtain the source tarball: use the offline override if given, otherwise
# download it from the pinned upstream URI. Either way it is hash-verified below.
# Locate the source tarball — expected in the current working directory.
# ---------------------------------------------------------------------------
if [ -n "${SRC_OVERRIDE}" ]; then
if [ ! -f "${SRC_OVERRIDE}" ]; then
echo "Error: source tarball not found: ${SRC_OVERRIDE}" >&2
exit 1
fi
SRC_TARBALL="$(realpath "${SRC_OVERRIDE}")"
echo "Source tarball -> ${SRC_TARBALL} (offline override)"
else
SRC_TARBALL="${WORKDIR}/${SRC_TARBALL_NAME}"
echo "Source tarball -> downloading ${SRC_TARBALL_URI}"
curl -fSL --retry 3 --retry-delay 2 -o "${SRC_TARBALL}" "${SRC_TARBALL_URI}"
SRC_TARBALL="${PWD}/${SRC_TARBALL_NAME}"
if [ ! -f "${SRC_TARBALL}" ]; then
echo "Error: source tarball not found: ${SRC_TARBALL}" >&2
echo "Place ${SRC_TARBALL_NAME} in the current working directory and re-run." >&2
exit 1
fi
echo "Source tarball -> ${SRC_TARBALL}"

# ---------------------------------------------------------------------------
# Verify the input against the hard-coded hash (accident prevention)
Expand All @@ -148,16 +110,12 @@ echo " OK (${SRC_TARBALL_SHA512})"
# ---------------------------------------------------------------------------
# Toolchain report — the documented "tolerance" for reproducibility
# ---------------------------------------------------------------------------
echo "Toolchain (canonical: go1.26.4 / tar (GNU tar) 1.35 / bzip2 1.0.8):"
echo " go -> $(go version 2>/dev/null || echo 'NOT FOUND')"
echo " tar -> $(tar --version | head -1)"
echo " bzip2 -> $(bzip2 --version 2>&1 | head -1)"
echo "Toolchain (canonical: go1.26.4):"
echo " go -> $(go version 2>/dev/null || echo 'NOT FOUND')"

pushd "${WORKDIR}" > /dev/null

NAME_VER="${PKG_NAME}-${PKG_VERSION}"
# Fedora forge macros expect the vendor archive as %{archivename}-vendor.tar.bz2.
VENDOR_TARBALL="${OUT_FOLDER}/${NAME_VER}-vendor.tar.bz2"

echo "Unpacking the source tarball."
tar -xf "${SRC_TARBALL}"
Expand All @@ -167,35 +125,10 @@ cd "${NAME_VER}"
echo "Getting the vendored modules."
go mod vendor

echo "Tar vendored modules (deterministic flags for reproducibility)."
# Pin every archived timestamp to the Unix epoch (@0 = 1970-01-01): --mtime sets it and
# --clamp-mtime caps anything newer. --mode normalizes permission bits (0644 files / 0755 dirs)
# so the hash is independent of the maintainer's umask. Combined with --sort=name, fixed
# owner/group and stripped pax atime/ctime, the tarball's hash depends solely on file content,
# independent of the host clock, locale, umask, or how the source was unpacked.
tar --sort=name \
--mode='go-w,u+rw,a+rX' \
--mtime="@0" --clamp-mtime \
--owner=0 --group=0 --numeric-owner \
--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime \
-cjf "${VENDOR_TARBALL}" vendor
echo "Moving vendor directory to output folder."
mkdir -p "${OUT_FOLDER}"
mv vendor "${OUT_FOLDER}/"

popd > /dev/null

# ---------------------------------------------------------------------------
# Verify the produced archive against the documented expected hash
# ---------------------------------------------------------------------------
ACTUAL_VENDOR_SHA512="$(sha512sum "${VENDOR_TARBALL}" | awk '{print $1}')"
echo "Telegraf vendored modules are available at (${VENDOR_TARBALL})."
echo "SHA512: ${ACTUAL_VENDOR_SHA512}"
if [ "${ACTUAL_VENDOR_SHA512}" = "${EXPECTED_VENDOR_SHA512}" ]; then
echo " MATCH: reproduces the pinned vendor archive."
else
echo " WARNING: does not match the pinned SHA512:" >&2
echo " expected: ${EXPECTED_VENDOR_SHA512}" >&2
echo " actual: ${ACTUAL_VENDOR_SHA512}" >&2
echo " This usually means a toolchain difference (the Go version writes" >&2
echo " vendor/modules.txt). Reproduce with the canonical toolchain above before" >&2
echo " uploading, or update EXPECTED_VENDOR_SHA512 (and telegraf.comp.toml) if this" >&2
echo " is an intentional change." >&2
fi
echo "Vendor directory is available at ${OUT_FOLDER}/vendor."
16 changes: 7 additions & 9 deletions base/comps/telegraf/telegraf.comp.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,10 @@ hash-type = "SHA512"
origin = { type = "download", uri = "https://github.com/influxdata/telegraf/archive/v1.38.2/telegraf-1.38.2.tar.gz" }

[[components.telegraf.source-files]]
filename = "telegraf-1.38.2-vendor.tar.bz2"
# Reproducible vendor archive (generate_source_tarball.sh / go_vendor_archive).
# Hosted in the AzL lookaside store (pkgs_modified) following the same scheme as other
# non-upstream sources, e.g. base/comps/espeak-ng/espeak-ng.comp.toml. The blob must be
# uploaded before CI source checks and package builds can fetch it; the merge queue's
# sources-upload pipeline publishes changed-component sources.
hash = "1108fe48086a7051c5cb89935c6de1c675c3ea8212a979d147ad0c03aef327c6234fa9eee292e4f9594ba9ec2cb757fc9eff46630aea43551bca3d948b30b27f"
hash-type = "SHA512"
origin = { type = "download", uri = "https://azltempstaginglookaside.blob.core.windows.net/repo/pkgs_modified/telegraf/telegraf-1.38.2-vendor.tar.bz2/sha512/1108fe48086a7051c5cb89935c6de1c675c3ea8212a979d147ad0c03aef327c6234fa9eee292e4f9594ba9ec2cb757fc9eff46630aea43551bca3d948b30b27f/telegraf-1.38.2-vendor.tar.bz2" }
filename = "telegraf-1.38.2-vendor.tar.gz"
origin.type = "custom"
origin.script = "generate_source_tarball.sh"
origin.mock-packages = ["golang"]
origin.inputs = ["telegraf-1.38.2.tar.gz"]
hash-type = "SHA512"
hash = "4004e684a8f65e404a4949eca7593b53981f7cf8a0622869b76dc694e6bffdd1be034ff460908277546160e9781ecd8fbfc8d86f096c57096d9ef286fcda086f"
2 changes: 1 addition & 1 deletion base/comps/telegraf/telegraf.spec
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ License: Apache-2.0 AND BSD-2-Clause AND BSD-2-Clause-Views AND BSD-3-Cla
URL: %{gourl}
Source0: %{gosource}
# Generated by go-vendor-tools
Source1: %{archivename}-vendor.tar.bz2
Source1: %{archivename}-vendor.tar.gz
Source2: go-vendor-tools.toml
# AzL: declarative system user (created at install time, before files are unpacked).
Source3: %{name}.sysusers
Expand Down
2 changes: 1 addition & 1 deletion locks/telegraf.lock
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Managed by azldev component update. Do not edit manually.
version = 1
input-fingerprint = 'sha256:a1987e5e45e9d18ca3fa9878959bc1ff101a4b1c2a61b7cf17e3de8f5f5581e7'
input-fingerprint = 'sha256:7837959a2d127f617cdd762692c1f16b60a6ca3d578951626703e062f72c2991'
2 changes: 1 addition & 1 deletion specs/t/telegraf/sources
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
SHA512 (telegraf-1.38.2.tar.gz) = 36c419978e98da9809e18865053399dd2198abc3d650f54424e3eb359ff8dfcb615f8b0a82dba484d03acfe5abe51a160a41ef486e5602f99400bd69e7afe48d
SHA512 (telegraf-1.38.2-vendor.tar.bz2) = 1108fe48086a7051c5cb89935c6de1c675c3ea8212a979d147ad0c03aef327c6234fa9eee292e4f9594ba9ec2cb757fc9eff46630aea43551bca3d948b30b27f
SHA512 (telegraf-1.38.2-vendor.tar.gz) = 4004e684a8f65e404a4949eca7593b53981f7cf8a0622869b76dc694e6bffdd1be034ff460908277546160e9781ecd8fbfc8d86f096c57096d9ef286fcda086f
7 changes: 5 additions & 2 deletions specs/t/telegraf/telegraf.spec
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
## (rpmautospec version 0.8.3)
## RPMAUTOSPEC: autorelease, autochangelog
%define autorelease(e:s:pb:n) %{?-p:0.}%{lua:
release_number = 2;
release_number = 3;
base_release_number = tonumber(rpm.expand("%{?-b*}%{!?-b:1}"));
print(release_number + base_release_number - 1);
}%{?-e:.%{-e*}}%{?-s:.%{-s*}}%{!?-n:%{?dist}}
Expand Down Expand Up @@ -45,7 +45,7 @@ License: Apache-2.0 AND BSD-2-Clause AND BSD-2-Clause-Views AND BSD-3-Cla
URL: %{gourl}
Source0: %{gosource}
# Generated by go-vendor-tools
Source1: %{archivename}-vendor.tar.bz2
Source1: %{archivename}-vendor.tar.gz
Source2: go-vendor-tools.toml
# AzL: declarative system user (created at install time, before files are unpacked).
Source3: %{name}.sysusers
Expand Down Expand Up @@ -206,6 +206,9 @@ install -d -m 0770 %{buildroot}%{_localstatedir}/lib/%{name}

%changelog
## START: Generated by rpmautospec
* Tue Jul 07 2026 Antonio Salinas <asalinas@microsoft.com> - 1.38.2-3
- feat(telegraf): Generate sources from scratch

* Thu Jul 02 2026 Autumn Nash <autumnnash@microsoft.com> - 1.38.2-2
- feat(telegraf): add curated telegraf 1.38.2 metrics agent for Azure Linux
4.0
Expand Down
Loading