Skip to content

feat(server): support RocksDB on RISC-V#3102

Open
Seanium wants to merge 1 commit into
apache:masterfrom
Seanium:feature/riscv64-rocksdb-support
Open

feat(server): support RocksDB on RISC-V#3102
Seanium wants to merge 1 commit into
apache:masterfrom
Seanium:feature/riscv64-rocksdb-support

Conversation

@Seanium

@Seanium Seanium commented Jul 21, 2026

Copy link
Copy Markdown

Purpose of the PR

Main Changes

  • Use checksum-pinned Alibaba Dragonwell 11 Extended only on RISC-V; amd64, arm/v7, arm64, ppc64le, and s390x continue to use eclipse-temurin:11-jre-jammy.
  • Install and automatically preload libatomic.so.1 before RocksDB JNI is used on RISC-V.
  • Upgrade architecture detection and use system protoc / grpc_java_plugin in automatically activated RISC-V Maven profiles.
  • Add an opt-in -Drocksdb-only reactor, distribution registry, and backend-boundary check. Docker and test helpers select it only for RISC-V; existing architectures retain the default full build.
  • Add JNI, REST, Gremlin, health, and restart-persistence smoke tests for amd64, arm64, and riscv64/QEMU.
  • Replace architecture-sensitive fixed waits in three Gremlin task tests with bounded completion/progress waits. No assertion or target test is skipped.
  • Document the supported ABI, Dragonwell runtime, QEMU/native workflows, limitations, and cleanup in hugegraph-server/README.md.

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • Default Java 11 build: 43/43 reactor modules passed; the distribution retained RocksDB, Cassandra, HBase, HStore, and the other existing backends.
    • Native RISC-V clean RocksDB-only build: 14/14 modules passed from a source tree without generated files.
    • Native RISC-V runtime: Dragonwell 11 Server VM; JNI put/get/reopen, REST schema/vertices/edge, Gremlin, health, and restart persistence passed.
    • Native RISC-V Core: 794 tests, 0 failures, 0 errors, 41 skipped.
    • Native RISC-V API: 161 tests, 0 failures, 0 errors, 14 skipped, with a non-empty valid JaCoCo XML report.
    • Final Docker runtime smoke passed for linux/amd64, linux/arm64, and linux/riscv64 under QEMU. amd64 and arm64 remained on Eclipse Temurin 11 and their full-backend distributions.
    • XML, workflow YAML, shell syntax, EditorConfig, Dockerfile checks for all three target architectures, and git diff --check passed.

Does this PR potentially affect the following parts?

  • Dependencies (RISC-V-only JDK/runtime libraries and protobuf/gRPC alignment; no new universal-distribution dependency)
  • Modify configurations
  • The public API
  • Other affects (Docker runtime image and CI runtime smoke)
  • Nope

Documentation Status

  • Doc - TODO
  • Doc - Done
  • Doc - No Need

Residual Risks

  • Native runtime validation used glibc 2.39; the documented glibc 2.30 lower bound is based on the packaged RocksDB JNI ELF symbol requirements.
  • No public statement was found that this exact Dragonwell 11 RISC-V binary passed Java SE TCK/JCK. Release qualification should still include sustained native stress and stability testing.
  • The published multi-platform image manifest and per-platform digests can only be verified after a release image is published; that remains a release gate and is not claimed by this PR.

@dosubot dosubot Bot added size:XXL This PR changes 1000+ lines, ignoring generated files. ci-cd Build or deploy feature New feature store Store module labels Jul 21, 2026
@Seanium
Seanium force-pushed the feature/riscv64-rocksdb-support branch from b5da4f4 to 25a16ff Compare July 21, 2026 13:59
@Seanium
Seanium force-pushed the feature/riscv64-rocksdb-support branch from 25a16ff to b9d9d8c Compare July 21, 2026 15:00

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The new multi-architecture gate cannot start under the repository policy, and the native/runtime validation paths retain correctness and liveness gaps. Evidence: exact-head workflow run 29841863669 ended startup_failure with zero jobs; static inspection of the changed workflow, test, and smoke scripts.


- name: Set up QEMU
if: matrix.arch != 'amd64'
uses: docker/setup-qemu-action@v3

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

‼️ Critical: the new multi-architecture validation cannot start

At this exact head, Docker Build CI run 29841863669 ended with startup_failure and zero jobs; its annotation reports that docker/setup-qemu-action@v3 and docker/setup-buildx-action@v3 are forbidden by the repository's Actions policy. Consequently none of the added amd64/arm64/riscv64 build and runtime smoke jobs execute. Please replace both setup actions with organization-allowed, commit-pinned actions or an allowed shell-based setup, or obtain explicit allowlisting, then require the complete matrix to pass.

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build HugeGraph Server for linux/${{ matrix.arch }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: this matrix does not exercise the native RISC-V Maven profiles

The Dockerfile runs Maven in a $BUILDPLATFORM stage, so on the amd64 runner the new arch=riscv64 profiles in hugegraph-core and hg-pd-grpc never activate. The runtime smoke can validate the packaged JNI library while the documented native path through /usr/bin/protoc, grpc_java_plugin, and the profile dependency override remains broken. Please add a native/QEMU RISC-V Maven gate, or install the system protobuf tools and explicitly build with -P riscv64-protobuf-tools -Drocksdb-only so these profiles are covered.

TimeUnit.SECONDS.toNanos(timeoutSeconds);
do {
HugeTask<Object> task = scheduler.task(id);
if (task.progress() >= expectedProgress) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: progress polling can accept an already-completed task

TaskAndResultScheduler.task() reloads persisted state, while TaskCallable.updateProgress() saves only after its 30-second default interval; this script normally finishes in about 2.2 seconds. Completion persists progress 10, and because this condition is checked before task.completed(), the helper can return and the test then tries to cancel a completed task, contradicting the expected CANCELLING/CANCELLED state. Please make intermediate progress observable (for example, set a short save interval in the job or use explicit synchronization) and reject completion before accepting the threshold.

exit 1
fi

JAVA_VERSION=$(java -version 2>&1 | awk -F '"' '/version/ {print $2; exit}')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: the native smoke test ignores the selected JAVA_HOME

The native instructions select Dragonwell via JAVA_HOME, and the server launcher honors that setting, but this script invokes bare java here, again for properties, and for the RocksDB JNI process. If another JVM precedes Dragonwell in PATH, the smoke test either fails spuriously or validates a different runtime from the server. Please resolve a single Java executable up front, preferring $JAVA_HOME/bin/java when set, verify it is executable, and use it for all three invocations.

wait_for_server() {
local attempt
for attempt in $(seq 1 240); do
if curl_request --silent --show-error --fail \

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: one stalled request can defeat the bounded readiness loop

curl_request supplies neither --connect-timeout nor --max-time. Curl's total request time is unlimited by default, so one accepted-but-stalled connection can prevent this 240-iteration loop from advancing; the normal API request helper has the same issue. This can consume the full 45-minute job timeout and delay cleanup and diagnostics. Please add explicit connection and total request timeouts to readiness probes and normal requests, sized separately if mutations need a longer bound.

@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 30.49%. Comparing base (89b648a) to head (b9d9d8c).

❗ There is a different number of reports uploaded between BASE (89b648a) and HEAD (b9d9d8c). Click for more details.

HEAD has 3 uploads less than BASE
Flag BASE (89b648a) HEAD (b9d9d8c)
4 1
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3102      +/-   ##
============================================
- Coverage     37.43%   30.49%   -6.94%     
+ Complexity      520      374     -146     
============================================
  Files           808      808              
  Lines         69581    69581              
  Branches       9165     9165              
============================================
- Hits          26048    21221    -4827     
- Misses        40750    45936    +5186     
+ Partials       2783     2424     -359     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd Build or deploy feature New feature size:XXL This PR changes 1000+ lines, ignoring generated files. store Store module

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

[Task] Support HugeGraph with RocksDB on RISC-V

2 participants