SRE-3988 ci: separate ucx lib - #18949
Draft
grom72 wants to merge 19 commits into
Draft
Conversation
|
Errors are Unable to load ticket data |
grom72
changed the title
Collaborator
|
Test stage Unit Test with memcheck completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18949/1/testReport/ |
Collaborator
|
Test stage Functional on EL 9 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-18949/6/testReport/ |
Restore automatic RPM runtime dependency generation in common packaging flow while keeping behavior limited to RPM output. - Add --rpm-autoreq back to build_package(). - Keep --rpm-autoprov enabled. - Apply both flags only when OUTPUT_TYPE=rpm. This avoids impacting non-RPM outputs and restores ELF-derived Requires metadata for RPM packages. This reverts changes introduced in #18057. Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Add explicit runtime dependency from libpmem-devel to
${pmem_lib} = ${pmdk_full}.
The devel package ships libpmem.so symlink but did not require the
runtime package providing libpmem.so.1, which can leave a broken
linker target when installed standalone.
This keeps devel/runtime package linkage consistent across RPM
outputs.
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Document rebuilds for defective RPM metadata in changelogs and bump non-PMDK package release numbers to force updates. Update release values in package_info.sh for: - daos-spdk - isa-l_crypto - mercury Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Introduce a dedicated RPM validation script and wire it into the packaging workflow.
What this commit delivers:
- add utils/rpms/verify_rpms.sh as a CI/local gate for generated RPM sets
- validate RPM integrity, payload readability, and dependency metadata consistency
- verify -devel lib*.so symlink linkage against runtime providers
- handle provider edge cases (single-line Provides output and payload-based fallback)
- keep dynamic ELF Requires validation for non-devel packages
- integrate execution from utils/rpms/build_packages.sh
- install required verification tooling via install scripts
- document script behavior, scope, and function-level contract comments
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
DAOS-19540 fix: guard RPM-only FPM options by output type
Limit RPM-specific FPM flags to RPM builds to avoid applying them to other package types.
Changes:
- In utils/rpms/fpm_common.sh, move --rpm-changelog under the same output_type==rpm block as --rpm-autoprov/--rpm-autoreq.
- In utils/rpms/libfabric.sh, guard --rpm-autoprov with OUTPUT_TYPE==rpm in both build paths.
Why:
- --rpm-* options are RPM-only and should not leak into non-RPM package flows.
- Keeps RPM behavior unchanged while making mixed output builds safer.
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Refactor code + improve doc + skip verify for no-rpm (deb) case.
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
DAOS-19540 fix: scope RPM autoreq and verification checks
Limit RPM verification to supported RPM platforms, restore a per-package noautoreq escape hatch for daos-client-tests-openmpi, and keep verifier checks aligned with explicit OpenMPI package dependencies.
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
DAOS-19540 fix: drop duplicate rpm-autoprov in libfabric.sh
--rpm-autoprov is already added in fpm_common.sh for all RPM builds.
Remove duplicate local additions from libfabric.sh to keep
the packaging flow simpler without changing behavior.
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
DAOS-19540 fix: reduce false positives in verify_rpms
Teach verify_rpms.sh to accept one-hop runtime package dependencies and
ignore base system library providers such as libc.so.6.
This fixes EL9 verifier failures for valid daos-devel and
daos-client-tests-openmpi packaging layouts.
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Fix to simplify the process
pm: add configurable RPM verification and simplify verify_rpms checks
- add verify toggle to build_packages.sh:
- new 2nd arg: verify (yes|no), default yes
- run verify_rpms.sh only when verify=yes
- add usage/docs header and license header (Google 2025, HPE 2026)
- add Docker build control for verification:
- new ARG DAOS_DEPS_VERIFY=yes in Dockerfile.leap.15
- pass DAOS_DEPS_VERIFY to build_packages.sh for deps and daos builds
- gate explicit verify_rpms.sh call with DAOS_DEPS_VERIFY
- simplify verify_rpms.sh:
- move to grep-based Requires matching
- remove unnecessary tooling dependency on file
- require libc soname presence via libc.so.6* entries for dynamic ELF binaries
- improve devel runtime dependency check to accept generated runtime naming variants
(e.g., libfoo1/libfoo0) at matching version
- intentionally exclude fused-devel from runtime-package check (fused runtime RPMs
are not built in this pipeline)
- remove pkg_versions map; use package presence map + on-demand rpm version query
- validated with syntax checks and containerized manual run against /home/daos/rpms/deps
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
rpm: fix devel-runtime validation and add DAOS-specific transitive allowance
- simplify verify_rpms runtime matching while keeping strict package integrity checks
- keep libc verification for dynamic ELF binaries based on libc.so.6* autoreq entries
- enforce direct -devel -> base runtime (= version) when base runtime package exists
- support distro runtime naming variants for lib* packages (libfoo-devel -> libfooN)
- exclude fused-devel from runtime-package check (fused runtime RPMs are intentionally not built)
- add narrow DAOS exception:
- allow daos-devel -> daos-client (= version)
- only when daos-client itself requires daos (= same version)
- remove noisy debug output and keep behavior deterministic across generated RPM sets
Also included in this series:
- add verify toggle to build_packages.sh: [build_type] [verify], default verify=yes
- add Docker build arg DAOS_DEPS_VERIFY=yes and wire it to package build/verify flow
- add license/usage header updates in build_packages.sh
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
The file RPM is only required by maldet. Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Skip-cancel-previous-bBuilds: true Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
In verify_rpms.sh, check_devel_runtime_dependency derives the runtime
package name by stripping -devel (e.g. isa-l_crypto-devel -> isa-l_crypto).
When the runtime package in the generated RPM set has a 'lib' prefix
(e.g. libisa-l_crypto), check_devel_runtime_dependency failed because
isa-l_crypto was missing from pkg_present and did not start with 'lib'.
Fix this by falling back to 'lib${runtime_pkg}' if the base runtime package
without the 'lib' prefix is not present in the generated RPM set.
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Skip-cancel-previous-builds: true
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Skip-test: true Skip-test-hardware: true
Skip-cancel-previous-builds: true Skip-test: true Skip-test-hardware: true Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Skip-cancel-previous-builds: true Skip-test: true Skip-test-hardware: true Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Skip-cancel-previous-builds: true
Skip-test: true
Skip-test-hardware: true
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Skip-cancel-previous-builds: true
Skip-test: true
Skip-test-hardware: true
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Skip-test: true Skip-test-hardware: true Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
grom72
force-pushed
the
grom72/SRE-3988-separate-ucx-lib
branch
from
August 28, 2026 19:21
57af1d1 to
910691c
Compare
Skip-test: true Skip-test-hardware: true Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
grom72
force-pushed
the
grom72/SRE-3988-separate-ucx-lib
branch
from
August 28, 2026 19:22
910691c to
5747dee
Compare
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com> Skip-test: true Skip-test-hardware: true Skip-cancel-previous-builds: true
Collaborator
|
Test stage Test RPMs on Leap 15 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18949/11/execution/node/610/log |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Steps for the author:
After all prior steps are complete: