[raffenet]

Pull Request Description

Add optional support for libcncoll collectives at the MPIR collective layer, covering blocking intracommunicator barrier, bcast, gather, gatherv, scatter, scatterv, allgather, allgatherv, alltoall, alltoallv, reduce, allreduce, and reduce_scatter_block.

Background and Comparison with #7998 (fi_collective)

PR #7998 prototyped collective offload through libfabric's fi_collective interface in ch4/ofi. While fi_collective makes sense when a provider offloads collectives directly to fabric hardware, using it for a host-driven collective library like libcncoll that executes schedules over point-to-point messaging introduces several unnecessary layers of indirection and artificial constraints:

1. Transport and device portability: In addition to its native libfabric FI_TAGGED transport backend, libcncoll can use MPI point-to-point communication as its transport. Integrating libcncoll at the MPIR collective layer over its MPI transport allows it to work over any MPICH device, netmod, or fabric (ch4:ofi with any provider, not just OPX, as well as ch4:ucx, ch3, shared memory, etc.), whereas the fi_collective path in ch4/ofi: Add support for fi_collective offload #7998 was restricted to ch4:ofi with a provider supporting FI_COLLECTIVE.
2. Collective selection and netmod boundary: After the removal of device-level collective hooks, MPIDI_OFI_coll_* sits below src/mpi/coll and is not directly visible to MPIR collective algorithm selection (coll_algorithms.txt / CVARs). Integrating libcncoll as an MPIR collective algorithm (MPIR_*_intra_cnc) makes it a first-class algorithm in MPICH's existing selection framework without reintroducing netmod collective hooks.
3. Communicator setup and fabric state: The fi_collective path requires requesting FI_COLLECTIVE domain/endpoint capabilities at MPI_Init, binding a dedicated event queue, and performing an asynchronous fi_av_set + fi_join_collective handshake on every communicator creation. In contrast, libcncoll initializes lazily over a private communicator copy only when a cnc collective is actually invoked.
4. Data path indirection: Routing through MPIDI_OFI -> fi_collective -> a libfabric utility/offload provider -> FI_TAGGED operations adds multiple translation layers (datatype/op mapping, completion queues, and progress polling) and single-NIC/single-VCI constraints. Calling libcncoll directly at the MPIR layer cuts out the fi_collective middleware entirely.

Implementation Summary

• Enabled at configure time with --with-cnc=<prefix> (disabled by default).
• Registers the cnc algorithm for all 13 blocking intracommunicator collectives supported by libcncoll (barrier-intra, bcast-intra, gather-intra, gatherv-intra, scatter-intra, scatterv-intra, allgather-intra, allgatherv-intra, alltoall-intra, alltoallv-intra, reduce-intra, allreduce-intra, and reduce_scatter_block-intra), selectable via the corresponding MPIR_CVAR_*_INTRA_ALGORITHM=cnc CVARs.
• Byte-oriented collectives (bcast, gather, gatherv, scatter, scatterv, allgather, allgatherv, alltoall, alltoallv) use contiguous host buffers in place and stage non-contiguous datatypes, negative-displacement segments, or device buffers locally via MPIR_Localcopy(..., MPIR_BYTE_INTERNAL) so ranks never diverge on collective eligibility.
• Reduction collectives (reduce, allreduce, reduce_scatter_block) match internal 32-bit integer and single-precision float datatype handles (MPIR_INT32, MPIR_FLOAT32 via MPIR_DATATYPE_GET_RAW_INTERNAL) with MPI_SUM, MPI_PROD, MPI_MIN, and MPI_MAX, staging device buffers locally when needed.
• On first eligible collective call on a communicator, lazily creates a private communicator copy (MPII_Comm_copy) and initializes a libcncoll MPI transport over it; both are released during communicator teardown in MPII_Coll_comm_cleanup.
• Guards the private communicator (CNC_PRIV_SKIP) so any internal collectives called during cnc_init() fall back to native MPIR algorithms rather than recursing, caches any transport initialization failure on the communicator, and falls back to existing MPIR collective algorithms when multi-threaded (MPI_THREAD_MULTIPLE) or when reduction datatypes/ops are not supported.
• Includes unit tests (test/units/cnc_coll.unit) with a lightweight libcncoll shim to exercise lazy initialization, transport reuse, cleanup, recursion avoidance, all 13 collective entry points, internal reduction datatype handles, local staging via MPIR_Localcopy, fallback paths, and cached error recovery in make unit-testing.

Author Checklist

• Provide Description
• Commits Follow Good Practice
• Passes All Tests
• Contribution Agreement

[raffenet]

Updated the branch to expand libcncoll coverage beyond bcast and allreduce to all nine fixed-size blocking intracommunicator collectives supported by libcncoll:

• barrier-intra (MPIR_Barrier_intra_cnc)
• bcast-intra (MPIR_Bcast_intra_cnc)
• gather-intra (MPIR_Gather_intra_cnc)
• scatter-intra (MPIR_Scatter_intra_cnc)
• allgather-intra (MPIR_Allgather_intra_cnc)
• alltoall-intra (MPIR_Alltoall_intra_cnc)
• reduce-intra (MPIR_Reduce_intra_cnc)
• allreduce-intra (MPIR_Allreduce_intra_cnc)
• reduce_scatter_block-intra (MPIR_Reduce_scatter_block_intra_cnc)

Also extended test/units/cnc_coll.unit and its shim headers to cover all nine entry points and their MPI_IN_PLACE / fallback cases.