Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 31 additions & 65 deletions dev/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ This file documents the release process for the "Rust Arrow Crates": `arrow`, `a
The Rust Arrow Crates are interconnected (e.g. `parquet` has an optional dependency on `arrow`) so we increment and release all of them together.

If any code has been merged to main that has a breaking API change, as defined
in [Rust RFC 1105] he major version number is incremented (e.g. `9.0.2` to `10.0.2`).
in [Rust RFC 1105] the major version number is incremented (e.g. `9.0.2` to `10.0.0`).
Otherwise the new minor version incremented (e.g. `9.0.2` to `9.1.0`).

[rust rfc 1105]: https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md
Expand All @@ -45,56 +45,50 @@ crates.io, the Rust ecosystem's package manager.

We create a `CHANGELOG.md` so our users know what has been changed between releases.

The CHANGELOG is created automatically using
[update_change_log.sh](https://github.com/apache/arrow-rs/blob/main/dev/release/update_change_log.sh)
## Prepare CHANGELOG and version

This script creates a changelog using github issues and the
labels associated with them.

## Prepare CHANGELOG and version:
- Ensure [`git-cliff`](https://git-cliff.org/docs/installation/) is installed

Now prepare a PR to update `CHANGELOG.md` and versions on `main` to reflect the planned release.

Do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)
First copy the contents of `CHANGELOG.md` into `CHANGELOG-old.md`.

Then do this in the root of this repository. For example [#2323](https://github.com/apache/arrow-rs/pull/2323)

```bash
git checkout main
git pull
git checkout -b <RELEASE_BRANCH>

# Update versions. Make sure to run it before the next step since we do not want CHANGELOG-old.md affected.
sed -i '' -e 's/14.0.0/39.0.0/g' `find . -name 'Cargo.toml' -or -name '*.md' | grep -v CHANGELOG.md | grep -v README.md`
# Update versions.
sed -i '' -e 's/14.0.0/39.0.0/g' `find . -name 'Cargo.toml' -or -name '*.md' | grep -v CHANGELOG.md | grep -v CHANGELOG-old.md | grep -v README.md`
cargo check # update Cargo.lock with new versions
git commit -a -m 'Update version'

# ensure your github token is available
export ARROW_GITHUB_API_TOKEN=<TOKEN>
# Assuming remote name is apache; if named differently ensure this is changed,
# since we need the tags from the GitHub repository
git fetch apache --tags

# manually edit ./dev/release/update_change_log.sh to reflect the release version
# create the changelog
./dev/release/update_change_log.sh
# commit the initial changes
git commit -a -m 'Create changelog'
# ensure your github token is available
export GITHUB_TOKEN=<TOKEN>

# run automated script to copy labels to issues based on referenced PRs
# (NOTE 1: this must be done by a committer / other who has
# write access to the repository)
#
# NOTE 2: this must be done after creating the initial CHANGELOG file
python dev/release/label_issues.py
# e.g. TAG=60.0.0; this is just used to format the CHANGELOG, so tag doesn't
# need to exist yet, and don't put an RC tag either
# (we are excluding our above version bump commit from the changelog)
git-cliff --tag <TAG> --unreleased --output CHANGELOG.md --skip-commit $(git rev-parse HEAD)

# review change log / edit issues and labels if needed, rerun, repeat as necessary
# note you need to revert changes to CHANGELOG-old.md if you want to rerun the script
./dev/release/update_change_log.sh
# review change log, adjust labels on PR's as required
# can rerun above git-cliff command if needed

# Commit the changes
git commit -a -m 'Update changelog'
# commit the changelog
git commit -a -m 'Create changelog'

git push
```

Note that when reviewing the change log, rather than editing the
`CHANGELOG.md`, it is preferred to update the issues and their labels
(e.g. add `invalid` label to exclude them from release notes)
`CHANGELOG.md`, it is preferred to update the PRs and their labels
(e.g. add `development-process` label to exclude them from release notes)

Merge this PR to `main` prior to the next step.

Expand All @@ -109,7 +103,7 @@ distribution servers.

Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, etc.

### Create git tag for the release:
### Create git tag for the release

While the official release artifact is a signed tarball, we also tag the commit it was created for convenience and code archaeology.

Expand Down Expand Up @@ -141,39 +135,12 @@ The `create-tarball.sh` script
apache distribution svn server

2. provide you an email template to
send to dev@arrow.apache.org for release voting.
send to <dev@arrow.apache.org> for release voting.

### Vote on Release Candidate tarball

Send an email, based on the output from the script to dev@arrow.apache.org. The email should look like

```
To: dev@arrow.apache.org
Subject: [VOTE][RUST] Release Apache Arrow

Hi,

I would like to propose a release of Apache Arrow Rust
Implementation, version 4.1.0.

This release candidate is based on commit: a5dd428f57e62db20a945e8b1895de91405958c4 [1]

The proposed release tarball and signatures are hosted at [2].
The changelog is located at [3].

Please download, verify checksums and signatures, run the unit tests,
and vote on the release.

The vote will be open for at least 72 hours.

[ ] +1 Release this as Apache Arrow Rust
[ ] +0
[ ] -1 Do not release this as Apache Arrow Rust because...

[1]: https://github.com/apache/arrow-rs/tree/a5dd428f57e62db20a945e8b1895de91405958c4
[2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-4.1.0
[3]: https://github.com/apache/arrow-rs/blob/a5dd428f57e62db20a945e8b1895de91405958c4/CHANGELOG.md
```
Send an email, based on the output from the script to <dev@arrow.apache.org>.
See an [example of how the email should look](https://lists.apache.org/thread/2vpxdt6n7kzo72sxpr7q8yyby4495gnk).

For the release to become "official" it needs at least three Apache Arrow PMC members to vote +1 on it.

Expand All @@ -189,7 +156,7 @@ The `dev/release/verify-release-candidate.sh` script in this repository can assi

If the release is not approved, fix whatever the problem is and try again with the next RC number

### If the release is approved,
### If the release is approved

Then, create a new release on GitHub using the tag `<version>` (e.g. `4.1.0`).

Expand All @@ -200,7 +167,7 @@ git tag <version> <version>-<rc>
git push apache <version>
```

Move tarball to the release location in SVN, e.g. https://dist.apache.org/repos/dist/release/arrow/arrow-rs-4.1.0/, using the `release-tarball.sh` script:
Move tarball to the release location in SVN, e.g. <https://dist.apache.org/repos/dist/release/arrow/arrow-rs-4.1.0/>, using the `release-tarball.sh` script:

```shell
./dev/release/release-tarball.sh 4.1.0 2
Expand All @@ -212,7 +179,7 @@ Congratulations! The release is now official!

The [`release.yml`] workflow automatically creates a github release for the tag.
Check that the release is created and contains the correct changelog here:
https://github.com/apache/arrow-rs/releases
<https://github.com/apache/arrow-rs/releases>

[`release.yml`]: https://github.com/apache/arrow-rs/blob/main/.github/workflows/release.yml#L1-L0

Expand Down Expand Up @@ -249,7 +216,6 @@ Rust Arrow Crates:
(cd arrow-ipc && cargo publish)
(cd arrow-csv && cargo publish)
(cd arrow-json && cargo publish)
(cd arrow-avro && cargo publish)
(cd arrow-arith && cargo publish)
(cd arrow-string && cargo publish)
(cd arrow-row && cargo publish)
Expand Down
65 changes: 38 additions & 27 deletions dev/release/create-tarball.sh
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,9 @@ else
tar=tar
fi

if ! git -C "${SOURCE_TOP_DIR}" rev-list --max-count=1 ${tag}; then
if ! commit_sha=$(git -C "${SOURCE_TOP_DIR}" rev-list --max-count=1 "${tag}"); then
echo "Cannot continue: unknown git tag: $tag"
exit 1
fi


Expand All @@ -78,6 +79,33 @@ url="https://dist.apache.org/repos/dist/dev/arrow/${release}-rc${rc}"

echo "Attempting to create ${tarball} from tag ${tag}"

# Write out the submodule commits referenced so that release verification
# can later ensure it uses the same versions of test data.
tmpdir=$(mktemp -d)
trap 'rm -r "${tmpdir}"' EXIT
git -C "${SOURCE_TOP_DIR}" ls-tree -r ${tag} \
| awk '$2 == "commit" { print $4, $3 }' > ${tmpdir}/.gitmodules-commits

# create <tarball> containing the files in git at $tag
# the files in the tarball are prefixed with {release}
# (e.g. apache-arrow-rs-4.0.1)
mkdir -p ${distdir}
(cd "${SOURCE_TOP_DIR}" && \
git archive --prefix ${release}/ \
--add-file ${tmpdir}/.gitmodules-commits ${tag} \
| gzip > ${tarball})

echo "Running rat license checker on ${tarball}"
${SOURCE_DIR}/run-rat.sh ${tarball}

echo "Signing tarball and creating checksums"
gpg --armor --output ${tarball}.asc --detach-sig ${tarball}
# create signing with relative path of tarball
# so that they can be verified with a command such as
# shasum --check apache-arrow-rs-4.1.0-rc2.tar.gz.sha512
(cd ${distdir} && shasum -a 256 ${tarname}) > ${tarball}.sha256
(cd ${distdir} && shasum -a 512 ${tarname}) > ${tarball}.sha512
sha256=$(cut -d ' ' -f 1 "${tarball}.sha256")

echo "Draft email for dev@arrow.apache.org mailing list"
echo ""
Expand All @@ -88,14 +116,17 @@ Subject: [VOTE][RUST] Release Apache Arrow Rust ${version} RC${rc}

Hi,

I would like to propose a release of Apache Arrow Rust Implementation, version ${tag}.
I would like to propose a release of Apache Arrow Rust Implementation, version ${version}.

This release candidate is based on commit: ${tag} [1]
This release candidate is based on commit: ${commit_sha} [1].
The SHA256 of the release candidate is: ${sha256}

The proposed release tarball and signatures are hosted at [2].

The changelog is located at [3].

The release tracking issue is: [5]

Please download, verify checksums and signatures, run the unit tests,
and vote on the release. There is a script [4] that automates some of
the verification.
Expand All @@ -106,35 +137,15 @@ The vote will be open for at least 72 hours.
[ ] +0
[ ] -1 Do not release this as Apache Arrow Rust ${version} because...

[1]: https://github.com/apache/arrow-rs/tree/${tag}
[1]: https://github.com/apache/arrow-rs/tree/${commit_sha}
[2]: ${url}
[3]: https://github.com/apache/arrow-rs/blob/${tag}/CHANGELOG.md
[3]: https://github.com/apache/arrow-rs/blob/${commit_sha}/CHANGELOG.md
[4]: https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
[5]: RELEASE_ISSUE
MAIL
echo "---------------------------------------------------------"



# create <tarball> containing the files in git at $tag
# the files in the tarball are prefixed with {release}
# (e.g. apache-arrow-rs-4.0.1)
mkdir -p ${distdir}
(cd "${SOURCE_TOP_DIR}" && \
git archive ${tag} --prefix ${release}/ \
| gzip > ${tarball})

echo "Running rat license checker on ${tarball}"
${SOURCE_DIR}/run-rat.sh ${tarball}

echo "Signing tarball and creating checksums"
gpg --armor --output ${tarball}.asc --detach-sig ${tarball}
# create signing with relative path of tarball
# so that they can be verified with a command such as
# shasum --check apache-arrow-rs-4.1.0-rc2.tar.gz.sha512
(cd ${distdir} && shasum -a 256 ${tarname}) > ${tarball}.sha256
(cd ${distdir} && shasum -a 512 ${tarname}) > ${tarball}.sha512

echo "Uploading to apache dist/dev to ${url}"
svn co --depth=empty https://dist.apache.org/repos/dist/dev/arrow ${SOURCE_TOP_DIR}/dev/dist
svn add ${distdir}
svn ci -m "Apache Arrow Rust ${version} ${rc}" ${distdir}
svn ci -m "Apache Arrow Rust ${version} ${rc}" ${distdir}
Loading
Loading