Skip to content

feat: adds top level engine crate for global orchestration - #24

Open
bmuddha wants to merge 4 commits into
programsfrom
engine
Open

feat: adds top level engine crate for global orchestration#24
bmuddha wants to merge 4 commits into
programsfrom
engine

Conversation

@bmuddha

@bmuddha bmuddha commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

What changed

Added the top-level magicblock-engine crate that wires keeper state,
transaction processing, block pacing, recovery, and MagicRoot account operations
behind the consumer-facing Engine handle.

Why

Embedding services need one entry point for opening durable state, reconstructing
it after recovery, submitting work, and coordinating shutdown.

Part of #4.

Closes #29.

Impact

  • Engine::account exposes committed create, update, patch, and delete
    operations; Engine::transaction supports execute, schedule, and simulation.
  • Instruction slices and Message values are signed with the engine authority
    and latest blockhash; sanitized views and encoded transactions are accepted
    without re-signing.
  • Startup restores keeper state, replays retained entries after a snapshot rewind,
    and starts the sequencer with internal or external block pacing.
  • Coordinated shutdown rejects new transactions, drains execution, and flushes
    durable or externally mirrored state according to the pacing source.

Reviewer notes

Replay quiesces at each sealed superblock and compares the reconstructed account
checksum with the recorded seal. Divergence returns
ReplayError::StateMismatch.

Follow-up

The replication crate uses the external pacer and replay paths to build followers
upstack.

Summary by CodeRabbit

  • New Features

    • Added a durable execution engine for account management, transaction processing, simulation, scheduling, and coordinated shutdown.
    • Added account creation, updates, patching, deletion, and Magic Root instruction support.
    • Added internal and externally controlled block pacing.
    • Added transaction composition with signing and validation.
    • Added recovery, replay, state validation, and restart support.
    • Added an optional test harness for engine workflows.
  • Documentation

    • Added engine usage and lifecycle documentation.
    • Updated account mutation examples and clarified patching, slot, and finalization behavior.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: bf12c03b-2cee-4097-b880-9536b7b86f76

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • ✅ Review completed - (🔄 Check again to review again)

Warning

.coderabbit.yaml has a parsing error

The CodeRabbit configuration file in this repository has a parsing error and default settings were used instead. Please fix the error(s) in the configuration file. You can initialize chat with CodeRabbit to get help with the configuration file.

💥 Parsing errors (1)
Validation error: Invalid input: expected boolean, received string at "reviews.auto_review.enabled"
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@engine/README.md`:
- Around line 41-42: Update the internal pacing description in the README to say
that it appends the reset marker at the current slot, matching the behavior of
BlockTicker::new and PaceMaker::spawn. Leave the surrounding volatile-account
and pacemaker task description unchanged.

In `@engine/src/accessor.rs`:
- Around line 79-81: Add a dedicated `EngineError::ShuttingDown` variant in
`engine/src/error.rs` with the existing shutdown message, then update both
`execute` and `schedule` in the accessors to return that typed variant instead
of constructing a string error. Ensure both shutdown checks use the same variant
so callers can match it directly.
- Around line 86-90: Update the public execute method around the time::timeout
call to make timeout semantics explicit: return a distinct timeout error that
callers can recognize and use to query the transaction signature, while
preserving the submitted transaction’s continued processing. Ensure timeout
handling is distinguishable from receive and execution errors rather than
converting all errors to strings.
- Around line 103-111: Update Accessor::simulate to check
self.engine.terminating before sending the SimulatorMessage::Transaction,
matching the guards in execute and schedule. Return the established shutdown
error immediately when termination has begun; otherwise preserve the existing
channel send and response handling.

In `@engine/src/lib.rs`:
- Around line 166-208: Ensure locally owned ShutdownManager instances terminate
spawned services on all error paths. In engine/src/lib.rs lines 166-208, scope
the replay body so the temporary engine is dropped, capture its result, then
signal the LedgerReplayer and await shutdown. In engine/src/testkit.rs lines
86-88, match the Engine::new result and await shutdown termination before
returning an error; preserve successful initialization behavior.
- Around line 146-155: Update the OwnedBlockstoreEntry::Superblock replay path
so a checksum mismatch cannot leave expected.id persisted as the sealed
superblock: either validate observed against expected before calling
accounts().set_superblock, or restore the previous sealed id before returning
ReplayError::StateMismatch. Preserve the existing barrier, sync, and mismatch
error behavior.

In `@engine/src/pacemaker.rs`:
- Around line 53-61: Validate that BlockstoreParams::blocktime is non-zero
before it reaches BlockTicker::new, preferably by using a non-zero duration type
or rejecting Duration::ZERO during configuration parsing. Preserve the existing
ticker initialization only for valid positive blocktimes, preventing
time::interval from receiving zero.
- Around line 129-158: Update the Pacer::run shutdown loop to acquire the next
boundary through a cancellable next_block operation, then execute handle for
that acquired block outside tokio::select! so shutdown cannot interrupt its
internal awaits or skip finalize_superblock. Replace the current pace-based flow
while preserving normal shutdown and error propagation behavior.

In `@engine/src/testkit.rs`:
- Around line 86-88: Update the Engine::new call in the test harness to handle
its error explicitly: if construction fails, call shutdown.terminate().await
before propagating the original error. Preserve the existing successful path
that obtains the current slot from the created engine.

In `@engine/src/transaction.rs`:
- Around line 90-121: Update magicblock to reject compiled messages whose
header.num_required_signatures exceeds the Engine authority’s single-signature
model. Validate this immediately after v1::Message::try_compile and return an
appropriate transaction error before constructing VersionedTransaction, ensuring
external-signer instructions cannot produce malformed data.

In `@engine/tests/recovery.rs`:
- Around line 58-59: Wrap the TestEngine::with(dirs, authority).await call in
the replay_aborts_on_checksum_mismatch test with the same timeout mechanism used
at Lines 90-95, preserving the existing reopen behavior while ensuring hangs
fail fast. Keep the timeout comment aligned with the implementation.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0b5637b3-1416-4d7a-ae44-c9aebbf23379

📥 Commits

Reviewing files that changed from the base of the PR and between 35c9834 and 5d3d9ee.

📒 Files selected for processing (15)
  • Cargo.toml
  • README.md
  • engine/Cargo.toml
  • engine/README.md
  • engine/src/accessor.rs
  • engine/src/error.rs
  • engine/src/lib.rs
  • engine/src/pacemaker.rs
  • engine/src/testkit.rs
  • engine/src/transaction.rs
  • engine/tests/accounts.rs
  • engine/tests/builtins.rs
  • engine/tests/recovery.rs
  • engine/tests/security.rs
  • engine/tests/transactions.rs

Comment thread engine/README.md
Comment on lines +41 to +42
Internal pacing appends one reset marker at the upcoming slot and clears
chain-mirrored volatile accounts before the pacemaker task starts. Internal

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Correct the reset slot description.

The README states that internal pacing "appends one reset marker at the upcoming slot". The code records the reset at the current slot. BlockTicker::new sets slot from engine.blocks().current_slot() at engine/src/pacemaker.rs Line 56, and PaceMaker::spawn calls engine.reset(ticker.slot) at Line 112 before the ticker advances. The doc comment at engine/src/pacemaker.rs Line 100 already says "current slot".

Align the README with the code.

📝 Proposed wording fix
-Internal pacing appends one reset marker at the upcoming slot and clears
+Internal pacing appends one reset marker at the current slot and clears
 chain-mirrored volatile accounts before the pacemaker task starts.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Internal pacing appends one reset marker at the upcoming slot and clears
chain-mirrored volatile accounts before the pacemaker task starts. Internal
Internal pacing appends one reset marker at the current slot and clears
chain-mirrored volatile accounts before the pacemaker task starts. Internal
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/README.md` around lines 41 - 42, Update the internal pacing
description in the README to say that it appends the reset marker at the current
slot, matching the behavior of BlockTicker::new and PaceMaker::spawn. Leave the
surrounding volatile-account and pacemaker task description unchanged.

Comment thread engine/src/accessor.rs
Comment on lines +79 to +81
if self.engine.terminating.load(Ordering::Acquire) {
Err("engine is shutting down".to_string())?;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use a typed shutdown error instead of a string.

execute and schedule both build the shutdown error from a String. Callers cannot distinguish shutdown from other engine failures without matching on message text. The tests already match on typed variants such as EngineError::TransactionExecution. Add a dedicated variant and reuse it in both call sites.

♻️ Proposed refactor

Add the variant in engine/src/error.rs:

#[error("engine is shutting down")]
ShuttingDown,

Then use it in both accessors:

     pub async fn execute(self) -> Result<TransactionResult<()>> {
         if self.engine.terminating.load(Ordering::Acquire) {
-            Err("engine is shutting down".to_string())?;
+            return Err(EngineError::ShuttingDown);
         }
     pub async fn schedule(self) -> Result<()> {
         if self.engine.terminating.load(Ordering::Acquire) {
-            Err("engine is shutting down".to_string())?;
+            return Err(EngineError::ShuttingDown);
         }

Also applies to: 95-97

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/accessor.rs` around lines 79 - 81, Add a dedicated
`EngineError::ShuttingDown` variant in `engine/src/error.rs` with the existing
shutdown message, then update both `execute` and `schedule` in the accessors to
return that typed variant instead of constructing a string error. Ensure both
shutdown checks use the same variant so callers can match it directly.

Comment thread engine/src/accessor.rs
Comment on lines +86 to +90
let status = time::timeout(EXECUTION_TIMEOUT, rx.recv())
.await
.map_err(|e| e.to_string())?
.map_err(|e| e.to_string())?;
Ok(status.result)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

A timeout does not cancel the submitted transaction.

execute sends the transaction to the sequencer, then awaits the signature subscription with a fixed 8-second bound. If the bound elapses, the function returns an error, but the transaction stays queued and can still commit. Callers that retry on this error can submit duplicate work.

Document this behavior on the public method, or return a distinct timeout error so callers can query the signature instead of retrying.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/accessor.rs` around lines 86 - 90, Update the public execute
method around the time::timeout call to make timeout semantics explicit: return
a distinct timeout error that callers can recognize and use to query the
transaction signature, while preserving the submitted transaction’s continued
processing. Ensure timeout handling is distinguishable from receive and
execution errors rather than converting all errors to strings.

Comment thread engine/src/accessor.rs
Comment on lines +103 to +111
pub async fn simulate(self) -> Result<TransactionResult<ExecutionRecord>> {
let (response, rx) = oneshot::channel();
let msg = SimulatorMessage::Transaction(Simulation {
transaction: self.transaction,
response,
});
self.engine.sequencer.simulation.send(msg).await?;
rx.await.map_err(Into::into)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Inspect terminating-flag usage and simulation channel lifecycle.
set -euo pipefail

rg -n -C 5 'terminating' --glob '*.rs'
rg -n -C 5 'simulation' --glob '*.rs' -g '!**/tests/**'

Repository: magicblock-labs/magicblock-engine

Length of output: 171


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Repo files matching accessor.rs:\n'
fd -a 'accessor\.rs$' . || true

printf '\nGit status and changed files summary:\n'
git status --short || true
git diff --stat || true

printf '\nSearch for accessors/simulator keywords broadly:\n'
rg -n 'simulate|execute|schedule|terminating|Simulation|SimulationMessage|sequencer|simulation' -g '*.rs' . || true

Repository: magicblock-labs/magicblock-engine

Length of output: 30640


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'engine/src/accessor.rs relevant section:\n'
sed -n '1,130p' engine/src/accessor.rs | cat -n

printf '\nprocessor/src/sequencer/mod.rs relevant methods:\n'
sed -n '140,205p' processor/src/sequencer/mod.rs | cat -n

printf '\nengine/src/lib.rs shutdown methods:\n'
sed -n '95,185p' engine/src/lib.rs | cat -n

printf '\nprocessor/src/simulator.rs spawning and lifecycle:\n'
sed -n '1,110p' processor/src/simulator.rs | cat -n

printf '\nProcessorError/sequitur send error types:\n'
rg -n 'enum .*Error|type Result =|TransactionAccessorError|Shutdown|terminating|simulation.*send|sequencer.*send' -g '*.rs' engine processor nucleus

Repository: magicblock-labs/magicblock-engine

Length of output: 25230


Check terminating before sending to the simulation channel.

execute and schedule reject submissions while terminating is set. simulate still sends SimulatorMessage::Transaction, so calls made after shutdown begins can hit the closed channel path and return a send error instead of the consistent shutdown error. Add the same self.engine.terminating guard before self.engine.sequencer.simulation.send(...).await?.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/accessor.rs` around lines 103 - 111, Update Accessor::simulate to
check self.engine.terminating before sending the SimulatorMessage::Transaction,
matching the guards in execute and schedule. Return the established shutdown
error immediately when termination has begun; otherwise preserve the existing
channel send and response handling.

Comment thread engine/src/lib.rs
Comment on lines +146 to +155
OwnedBlockstoreEntry::Superblock(expected) => {
let _guard = self.barrier().await?;
self.accounts().set_superblock(expected.id);
self.sync(false)?;
let observed = self.superblocks().sealed();
if observed != expected {
error!(?observed, ?expected, "state mismatch; aborting replay");
Err(ReplayError::StateMismatch)?;
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Locate set_superblock and the sealed/position accessors used by the replay arm.
rg -n -C8 'fn set_superblock' --type=rust
rg -n -C8 'fn sealed\b' --type=rust
rg -n -C10 'fn sync\s*\(' --type=rust -g '!engine/**'

# Inspect the keeper startup decision that consumes the persisted sealed id.
fd -t f -e rs . keeper | xargs rg -n -C10 'superblock' -g '!**/tests/**'

Repository: magicblock-labs/magicblock-engine

Length of output: 171


🏁 Script executed:

#!/bin/bash
set -u

echo "Repository root files:"
git ls-files | sed -n '1,120p'

echo
echo "Locate engine/src/lib.rs:"
git ls-files | rg '(^|/)engine/src/lib\.rs$|engine/README.md|keeper|accountsdb|superblock|sealed' || true

echo
echo "Search set_superblock:"
rg -n -C6 'set_superblock' --type=rust . || true

echo
echo "Search sealed superblock related definitions/usages:"
rg -n -C8 'sealed|superblock|sync\s*\(' --type=rust . || true

Repository: magicblock-labs/magicblock-engine

Length of output: 50390


🏁 Script executed:

#!/bin/bash
set -u

echo "engine/src/lib.rs replay and accessor context:"
sed -n '1,230p' engine/src/lib.rs

echo
echo "keeper/src/builder.rs accountsdb/open and recovery context:"
sed -n '240,330p' keeper/src/builder.rs

echo
echo "accountsdb/src/lib.rs metadata/accessors and persisted store:"
sed -n '1,220p' accountsdb/src/lib.rs

echo
echo "accountsdb/src/store/mmap.rs metadata read/write/persist context:"
sed -n '120,210p' accountsdb/src/store/mmap.rs

echo
echo "Find persisted.write/flush implementations:"
sed -n '1,260p' accountsdb/src/store/kv.rs

Repository: magicblock-labs/magicblock-engine

Length of output: 25879


🏁 Script executed:

#!/bin/bash
set -u

echo "accountsdb/src/store/mod.rs flush and persisted store implementation:"
rg -n -C8 'struct PersistedStore|flush\(|defragment|write\(|meta\(\)' accountsdb/src/store/mod.rs

echo
echo "accountsdb/src/tests.rs relevant test for StateMismatch:"
rg -n -C20 'replay_aborts_on_checksum_mismatch|StateMismatch|set_superblock|corrupt' accountsdb/src/tests.rs engine/tests/recovery.rs

echo
python3 - <<'PY'
from pathlib import Path
for path in Path('.').glob('**/*.rs'):
    text = path.read_text(errors='ignore')
    if 'super_block: u64' in text or 'superblock: u64' in text or 'fn sealed' in text or 'super_block' in text:
        hits = [i+1 for i,line in enumerate(text.splitlines()) if 'superblock' in line or 'sealed' in line or 'super_block' in line]
        if hits and path != Path('accountsdb/src/store/mod.rs'):
            print(path, hits[:20])
PY

Repository: magicblock-labs/magicblock-engine

Length of output: 13892


🏁 Script executed:

#!/bin/bash
set -u

echo "Locate superblock sealed accessors in ledger packages:"
rg -n -C8 '\bsealed\b|\bsuperblock\b' ledger nucleus accountsdb keeper engine --type rust | rg -n -C8 'sealed'

echo
echo "Ledger package files:"
git ls-files ledger nucleus accountsdb -- '*ledger*' '*schema*' '*sealed*' '*blockstore*' | sed -n '1,200p'

echo
echo "Read ledger src files candidates relevant to OwnedBlockstoreEntry:"
for f in $(git ls-files ledger/src | sed -n '1,120p'); do
  echo "--- $f"
  grep -n "OwnedBlockstoreEntry\\|sealed\\|superblock" "$f" | sed -n '1,120p' || true
done

Repository: magicblock-labs/magicblock-engine

Length of output: 50389


Revert the superblock stamp after a replay checksum mismatch.

set_superblock(expected.id) writes accountsdb/current/.../mmap.meta, and flush(false) can leave that value in the durable store before the next replay starts. If observed != expected, the later open reads a sealed superblock that the accountsdb contents did not winnow through. Set the superblock only after the checksum matches, or restore the previous sealed id before returning StateMismatch.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/lib.rs` around lines 146 - 155, Update the
OwnedBlockstoreEntry::Superblock replay path so a checksum mismatch cannot leave
expected.id persisted as the sealed superblock: either validate observed against
expected before calling accounts().set_superblock, or restore the previous
sealed id before returning ReplayError::StateMismatch. Preserve the existing
barrier, sync, and mismatch error behavior.

Comment thread engine/src/pacemaker.rs
Comment on lines +53 to +61
impl BlockTicker {
/// Builds an interval ticker starting at the engine's current slot.
pub(crate) fn new(engine: &Engine, blocktime: Duration) -> Self {
let slot = engine.blocks().current_slot();
let mut ticker = time::interval(blocktime);
ticker.set_missed_tick_behavior(MissedTickBehavior::Skip);
ticker.reset();
BlockTicker { slot, ticker }
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Locate BlockstoreParams and any validation of blocktime.
rg -n -C12 'struct BlockstoreParams' --type=rust
rg -n -C6 '\bblocktime\b' --type=rust

Repository: magicblock-labs/magicblock-engine

Length of output: 171


🌐 Web query:

tokio time::interval panics zero duration period

💡 Result:

In Tokio, both the tokio::time::interval and tokio::time::interval_at functions will panic if the provided period duration is zero [1][2][3][4]. This behavior is implemented as an explicit assertion: assert!(period > Duration::new(0, 0), "period must be non-zero."); within the source code [2][3]. Ensuring a non-zero duration is necessary for the interval mechanism to function correctly, as it defines the time step between consecutive ticks [1][5].

Citations:


🏁 Script executed:

#!/bin/bash
set -u

printf 'files:\n'
git ls-files | sed -n '1,160p'

printf '\nfind pacemaker:\n'
git ls-files | rg '/pacemaker\.rs$|pacemaker' || true

printf '\nsearch BlockstoreParams/blocktime/ticker:\n'
rg -n -C12 'BlockstoreParams|blocktime|BlockTicker|time::interval|new\s*\(' --type=rust . || true

printf '\nfind Cargo deps for tokio time:\n'
git ls-files 'Cargo.toml' 'Cargo.lock' | xargs rg -n 'tokio|time' || true

Repository: magicblock-labs/magicblock-engine

Length of output: 50390


🏁 Script executed:

#!/bin/bash
set -u

printf 'Find BlockstoreParams files:\n'
rg -n "struct BlockstoreParams|impl BlockstoreParams|BlockstoreParams" --glob '*.rs' .

printf '\nCandidate files containing BlockstoreParams:\n'
files="$(rg -l "struct BlockstoreParams" --glob '*.rs' . | sort -u)"
printf '%s\n' "$files"

for f in $files; do
  echo "===== $f ====="
  wc -l "$f"
  ast-grep outline "$f" --match BlockstoreParams --view expanded || true
  rg -n -C20 "BlockstoreParams|blocktime|duration|Duration|superblock|NonZeroU64|serde" "$f" || true
done

Repository: magicblock-labs/magicblock-engine

Length of output: 4378


🏁 Script executed:

#!/bin/bash
set -u

printf 'Files mentioning BlockstoreParams construction / deserialization:\n'
rg -l "BlockstoreParams" --glob '*.rs' .

printf '\nDetailed occurrences without context limit:\n'
rg -n "BlockstoreParams|blocktime|Duration|from_str|parse|deserialize|serde" --glob '*.rs' .

printf '\nEngine/keeper configs and tests around zero blocktime:\n'
rg -n -C5 "blocktime|0\.0|0|BlockstoreParams" --type-config --type-toml --glob '*.rs' --glob '*.toml' --glob '*.json' .

Repository: magicblock-labs/magicblock-engine

Length of output: 18785


Accept a non-zero blocktime before creating the ticker.

BlockstoreParams::blocktime is a raw Duration, so deserialization can produce Duration::ZERO for config values such as 0s. That zero value reaches BlockTicker::new, and Tokio panics on time::interval(Duration::ZERO). Type this field as non-zero or validate it during config parsing before starting the pacemaker.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/pacemaker.rs` around lines 53 - 61, Validate that
BlockstoreParams::blocktime is non-zero before it reaches BlockTicker::new,
preferably by using a non-zero duration type or rejecting Duration::ZERO during
configuration parsing. Preserve the existing ticker initialization only for
valid positive blocktimes, preventing time::interval from receiving zero.

Comment thread engine/src/pacemaker.rs
Comment on lines +129 to +158
async fn run(mut self, mut shutdown: ShutdownHandle) {
let mut res = loop {
tokio::select! {
biased;
_ = shutdown.signalled() => {
break Ok(());
}
success = self.pace() => {
if !*success.as_ref().unwrap_or(&false) {
break success.map(|_| ());
}
}
}
};
res = if let Pacer::Internal(ref mut t) = self.pacer {
// Await every shutdown step even after an earlier failure.
let b = t.block();
res.and(self.handle(b).await).and(self.shutdown(false).await)
} else {
res.and(self.shutdown(true).await)
};
// Release engine storage before the manager can reopen it.
drop(self);
if let Err(error) = res {
error!(?error, "pace maker terminated with critical failure");
shutdown.terminate(ShutdownReason::Error(error.into()));
} else {
shutdown.terminate(ShutdownReason::Signalled);
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🔴 Critical | 🏗️ Heavy lift

Shutdown can cancel pace() in the middle of a block boundary.

tokio::select! uses biased, so the shutdown branch is always polled first. When shutdown fires, the self.pace() future is dropped at whatever await point it reached. handle is not a single atomic step. It performs three separate awaits at Lines 192-195.

Two partial states are reachable:

  • Cancellation between Line 192 and Line 193 advances the execution environment to block but leaves the simulation environment behind.
  • Cancellation between Line 195 and Line 196 drops the barrier and skips finalize_superblock() after execution already crossed the superblock boundary. The sealed superblock then trails the execution state.

Per engine/README.md Lines 29-31, a sealed superblock that trails the retained ledger makes keeper restore a snapshot and replay on the next open. A clean shutdown therefore turns into a recovery path, and the seal that handle exists to guarantee is skipped.

Acquire the next boundary inside select!, then run handle outside it, so a started boundary always completes.

🔒 Proposed fix: only the block acquisition is cancellable
+    /// Waits for the next boundary. Cancellation-safe.
+    ///
+    /// Returns `None` when an external block source is exhausted.
+    async fn next_block(&mut self) -> Option<(Block, Option<oneshot::Sender<()>>)> {
+        match &mut self.pacer {
+            Pacer::Internal(t) => {
+                t.ticker.tick().await;
+                Some((t.block(), None))
+            }
+            Pacer::External(rx) => {
+                let msg = rx.recv().await?;
+                Some((msg.block, Some(msg.submitted)))
+            }
+        }
+    }
+
     async fn run(mut self, mut shutdown: ShutdownHandle) {
         let mut res = loop {
-            tokio::select! {
+            let next = tokio::select! {
                 biased;
                 _ = shutdown.signalled() => {
-                    break Ok(());
+                    break Ok(());
                 }
-                success = self.pace() => {
-                    if !*success.as_ref().unwrap_or(&false) {
-                        break success.map(|_| ());
-                    }
-                }
-            }
+                next = self.next_block() => next,
+            };
+            // Run the boundary to completion; shutdown must not split it.
+            let Some((block, submission)) = next else {
+                break Ok(());
+            };
+            let result = self.handle(block).await;
+            if let Some(submission) = submission
+                && result.is_ok()
+            {
+                let _ = submission.send(());
+            }
+            if let Err(error) = result {
+                break Err(error);
+            }
         };

Remove pace once next_block replaces it, or keep pace and make handle the only non-cancellable part.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/pacemaker.rs` around lines 129 - 158, Update the Pacer::run
shutdown loop to acquire the next boundary through a cancellable next_block
operation, then execute handle for that acquired block outside tokio::select! so
shutdown cannot interrupt its internal awaits or skip finalize_superblock.
Replace the current pace-based flow while preserving normal shutdown and error
propagation behavior.

Comment thread engine/src/testkit.rs
Comment on lines +86 to +88
let mut shutdown = ShutdownManager::default();
let engine = Engine::new(builder, rx, &mut shutdown).await?;
let slot = engine.blocks().current_slot();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

A failed Engine::new leaves the harness services running.

Line 86 creates the ShutdownManager. Line 87 passes it to Engine::new and propagates the error with ?. On that path the manager is dropped without terminate().await, so every service that Engine::new already registered and spawned keeps running and keeps the Dirs temp directories open.

engine/tests/recovery.rs::replay_aborts_on_checksum_mismatch takes this path deliberately at Line 92. The leaked services outlive the assertion for the rest of the test binary.

Terminate the manager before you propagate the error.

🧹 Proposed fix: terminate the manager on the failure path
         let mut shutdown = ShutdownManager::default();
-        let engine = Engine::new(builder, rx, &mut shutdown).await?;
+        let engine = match Engine::new(builder, rx, &mut shutdown).await {
+            Ok(engine) => engine,
+            Err(error) => {
+                shutdown.terminate().await;
+                return Err(error);
+            }
+        };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let mut shutdown = ShutdownManager::default();
let engine = Engine::new(builder, rx, &mut shutdown).await?;
let slot = engine.blocks().current_slot();
let mut shutdown = ShutdownManager::default();
let engine = match Engine::new(builder, rx, &mut shutdown).await {
Ok(engine) => engine,
Err(error) => {
shutdown.terminate().await;
return Err(error);
}
};
let slot = engine.blocks().current_slot();
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/testkit.rs` around lines 86 - 88, Update the Engine::new call in
the test harness to handle its error explicitly: if construction fails, call
shutdown.terminate().await before propagating the original error. Preserve the
existing successful path that obtains the current slot from the created engine.

Comment thread engine/src/transaction.rs
Comment on lines +90 to +121
/// Composes an Engine-private transaction and signs its final
/// Magicblock wire representation with the Engine authority.
pub(crate) fn magicblock(instructions: &[Instruction], engine: &Engine) -> Result<Vec<u8>> {
let message = v1::Message::try_compile(&engine.authority(), instructions, engine.blockhash())?;
let message = VersionedMessage::V1(message);
// These checks are merely future proof defenses, currently it should be
// impossible to construct a transaction which might violate any of them
if message.instructions().len() > MAGICBLOCK_INSTRUCTION_TRACE_LENGTH {
Err(TransactionError::SanitizeFailure)?;
} else if message.static_account_keys().len() > MAX_MAGICBLOCK_ACCOUNT_LOCKS {
Err(TransactionError::TooManyAccountLocks)?;
}
for ix in message.instructions() {
if ix.accounts.len() > MAX_MAGICBLOCK_ACCOUNT_LOCKS {
Err(TransactionError::TooManyAccountLocks)?;
}
}

// Reserve the trailing signature slot without signing the V1 prefix, which
// is replaced below before the only signing operation.
let transaction = VersionedTransaction {
signatures: vec![Default::default()],
message,
};
let mut data = wincode::serialize(&transaction).map_err(wincode::Error::from)?;
// Patch the transaction prefix to allow for larger tranaction limits
data[0] = MAGICBLOCK_PREFIX;

let signature_offset = data.len() - SIGNATURE_SIZE;
let signature = engine.signer().sign_message(&data[..signature_offset]);
data[signature_offset..].copy_from_slice(signature.as_ref());
Ok(data)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate transaction.rs and relevant files =="
fd -a 'transaction\.rs$' . | sed 's#^\./##' | head -20

echo "== inspect candidate file outline =="
ast-grep outline engine/src/transaction.rs --view expanded || true

echo "== relevant magicblock section =="
sed -n '1,180p' engine/src/transaction.rs | cat -n

echo "== search signature/header usage =="
rg -n "num_required_signatures|magicblock|VersionedTransaction|sign_message|Message::try_compile|MAGICBLOCK_PREFIX|MAX_MAGICBLOCK_ACCOUNT_LOCKS|MAGICBLOCK_INSTRUCTION_TRACE_LENGTH" engine/src -S

Repository: magicblock-labs/magicblock-engine

Length of output: 8284


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== dependency manifests =="
for f in Cargo.toml engine/Cargo.toml solana/Cargo.toml; do
  if [ -f "$f" ]; then
    echo "--- $f ---"
    rg -n 'solana-(message|transaction|signer|instruction)|agave-transaction-view|wincode' "$f" -C 2
  fi
done

echo "== source files containing VersionedTransaction/try_compile/serialize =="
rg -n "struct VersionedTransaction|impl Serialize for VersionedTransaction|fn try_compile|MessageHeader|num_required_signatures|serialize\\(" engine Cargo.toml engine/Cargo.toml solana -S --glob '!**/target/**' | head -200

echo "== inspect dependency source if vendored =="
for base in engine/solana crates solana; do
  if [ -d "$base" ]; then
    echo "--- $base ---"
    fd -a 'versioned.*\.rs|signature.*\.rs|v1.*\.rs|message.*\.rs' "$base" | head -100
  fi
done

Repository: magicblock-labs/magicblock-engine

Length of output: 25284


🌐 Web query:

solana v1::Message::try_compile num_required_signatures SignerMeta header trailing signatures

💡 Result:

In the Solana transaction architecture, Message::try_compile is a method used to construct a versioned transaction message from instructions, a payer, and address lookup table accounts [1][2][3]. The MessageHeader structure, common to legacy, v0, and v1 transaction formats, is critical for determining how transaction signatures and accounts are processed [4][5]. Its fields are: 1. num_required_signatures (u8): Specifies the number of signatures required for the transaction to be considered valid [6][7]. The signers must correspond to the first num_required_signatures elements of the message's account_keys array [6][4][8]. 2. num_readonly_signed_accounts (u8): Defines how many of the signed accounts are read-only [6][4]. 3. num_readonly_unsigned_accounts (u8): Defines how many of the unsigned accounts are read-only [6][4]. Regarding signatures and trailing data: - Signatures: A transaction must contain exactly num_required_signatures signatures [9][8]. Each signature corresponds to the public key at the same index in the message's account_keys [9][8]. - Trailing Data: Solana transaction protocols, particularly in newer formats like v1, strictly prohibit trailing data after the signatures or the message structure [9]. Versioned transactions (such as v0 and the proposed v1) utilize a version prefix byte to distinguish themselves from legacy transactions, ensuring parsers correctly identify the structure and prevent issues related to unexpected trailing data [5]. The V1 transaction format, currently under development, aims to increase transaction size limits and improve efficiency by embedding compute budget configurations directly into the header [10]. SignerMeta and related structures help organize these accounts based on their permissions (writable/read-only and signer/non-signer) to ensure efficient signature verification and transaction processing [6][4].

Citations:


🌐 Web query:

solana transaction VersionedTransaction serialize signatures trailing signatures num_required_signatures

💡 Result:

In Solana, a VersionedTransaction is serialized by encoding a compact-u16 length prefix of the number of signatures, followed by the concatenated array of 64-byte Ed25519 signatures, and finally the serialized message bytes [1][2][3]. The transaction does not allow trailing data after the signatures and message fields [4]. The num_required_signatures value is a property defined within the MessageHeader of the transaction message [1][4]. It specifies the number of accounts that must sign the transaction to be considered valid [1]. During serialization and sanitization processes, the system validates that the number of provided signatures matches this num_required_signatures count exactly [5]. In the @solana/web3.js library, the VersionedTransaction handles this by indexing signatures corresponding to the public keys found in the first num_required_signatures positions of the static account keys array [2][3]. If the signature count does not match the required number, the transaction will fail validation [5].

Citations:


🌐 Web query:

github solana-labs solana solana-transaction VersionedTransaction wincode serialize

💡 Result:

In the Solana JavaScript/TypeScript SDK (@solana/web3.js), the VersionedTransaction class provides a serialize method that returns a Uint8Array containing the transaction in its wire format [1]. This serialized format is what you transmit to the Solana cluster or share between clients [2][3][4]. To serialize a VersionedTransaction, you call the serialize method directly on the transaction instance: const serializedTransaction = transaction.serialize; Key points regarding serialization and VersionedTransaction: 1. Serialization Output: The serialize method returns a Uint8Array [1]. If you need this in a format suitable for network transmission (e.g., base64 string), you can convert the result (e.g., Buffer.from(transaction.serialize).toString('base64')) [2]. 2. Signing Requirements: Unlike legacy transactions, a VersionedTransaction must be fully signed before you send it to the cluster via connection.sendTransaction [3][4]. You can sign the transaction using the transaction.sign method, passing an array of required Signers [3][4]. 3. Deserialization: To reconstruct a transaction from a serialized byte array, use the static VersionedTransaction.deserialize(buffer) method [1][2]. 4. Rust Implementation: In the Rust solana-sdk, VersionedTransaction implements the Serialize trait (from the serde crate), allowing it to be serialized into various formats for internal or off-chain use, though the standard wire-format serialization is handled through the transaction-specific implementation [5][6][7]. For further implementation details and code examples, consult the official Solana documentation on versioned transactions [3][4].

Citations:


🌐 Web query:

github solana-labs solana solana-transaction VersionedTransaction deserialize num_required_signatures signatures

💡 Result:

In the Solana codebase, a VersionedTransaction is a structure composed of a list of signatures and a message [1][2]. The relationship between these signatures and the num_required_signatures field found in the transaction message header is enforced during the sanitization and validation stages of the transaction pipeline [3][4]. 1. Structure: The VersionedTransaction struct contains a Vec (in Rust) or an array of Uint8Array (in JavaScript/TypeScript) and a VersionedMessage [5][1][6]. The signatures are stored as a compact-encoded array [7][6]. 2. Validation Logic: When a transaction is deserialized and subsequently sanitized (e.g., via the sanitize_signatures method in the Solana Rust SDK), the number of signatures provided is compared against the num_required_signatures specified in the message header [3]. Specifically: - The num_required_signatures is extracted from the message header [3]. - The logic requires that the number of signatures matches exactly the num_required_signatures [3]. - If the number of provided signatures is less than the number required, it is typically treated as an error (e.g., SanitizeError::InvalidValue or similar depending on the specific check) [3]. - If there are more signatures than required, this is also often treated as an error, as the system expects a strict correspondence between the expected signers and the provided signatures [3][4]. 3. Serialization/Deserialization: During deserialization (e.g., VersionedTransaction::deserialize in the web3.js library), the transaction bytes are parsed to first extract the signatures (using compact-u16 length encoding) and then the message [5][8]. The message header, which contains the num_required_signatures, is part of the VersionedMessage and is used later in the pipeline to verify the transaction [7][4]. In summary, num_required_signatures dictates how many signatures the transaction must contain to be considered valid; the deserialization process prepares the signatures array, and the sanitization process ensures the count in that array matches the requirement set by the transaction header [3][4].

Citations:


🌐 Web query:

github solana-labs solana solana-messagev1 Message::try_compile num_required_signatures

💡 Result:

In the Solana codebase, Message::try_compile is a method used to create a versioned transaction message (specifically V0) from a payer, a set of instructions, address lookup table accounts, and a recent blockhash [1][2]. The resulting message contains a MessageHeader struct [3]. The num_required_signatures field within the MessageHeader is a u8 value that defines the number of signatures required for the message to be considered valid [4][5][3]. When a transaction is processed, the system requires that the signers of the provided signatures match the first num_required_signatures of the account_keys array stored within the message [5][3]. Essentially, this field serves as a partition to identify which accounts in the transaction's account list must have provided a valid signature for the transaction to proceed [5]. The try_compile function automatically calculates and populates this and other header fields based on the provided instructions and account lookup tables to ensure the resulting message is properly formed for sign-verification [2][6].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

cargo --version
cargo tree -p solana-transaction --format'{p}: {f}' 2>/dev/null | head -80 || true

echo "== find local crate source from cargo metadata if present =="
cargo metadata --format-version 1 --no-deps 2>/dev/null jq -r '.packages[] | select(.name=="solana-transaction" or .name=="solana-message") | "\(.name) \(.manifest_path) \(.version)"' 2>/dev/null || true

echo "== inspect wincode usages in transaction-view =="
sed -n '1,130p' solana/transaction-view/src/transaction_frame.rs | cat -n
sed -n '1,90p' solana/transaction-view/src/signature_frame.rs | cat -n

Repository: magicblock-labs/magicblock-engine

Length of output: 10931


🌐 Web query:

docs.rs solana-transaction 4.1.1 source versioned mod.rs

💡 Result:

The source code for the versioned module in the solana-transaction crate, specifically version 4.1.1, can be accessed through the official docs.rs platform [1]. The versioned module is defined in the file mod.rs within the versioned directory of the crate [1]. This module provides the implementation for the VersionedTransaction struct [1], which supports transactions containing multiple versions of messages [2][1]. Key components of the versioned/mod.rs file include: - The Definition of VersionedTransaction: A struct representing an atomic transaction, containing a list of signatures and a message of type VersionedMessage [3][1]. - Essential Methods: - try_new: A method for signing a versioned message and returning a signed transaction [1]. - sanitize: A function to perform validation on the transaction [1]. - version: Returns the specific version of the transaction (e.g., legacy, v0) [3][1]. - into_legacy_transaction: Allows converting the transaction into a legacy transaction if the underlying message is legacy [3][1]. - Crate Integration: The versioned module is declared in the main library file (lib.rs) of the solana-transaction crate [4]. You can view the source directly at the following URL: https://docs.rs/solana-transaction/4.1.1/src/solana_transaction/versioned/mod.rs.html [1]. Note that as of August 2026, newer versions (such as 4.1.2 and 4.1.3) are available, but the structure remains consistent with the 4.1.1 implementation [5][6].

Citations:


🌐 Web query:

github solana-labs solana sdk/src/transaction/versioned/mod.rs wincode serialize

💡 Result:

In the Solana codebase, wincode is used as the serialization framework for VersionedTransaction, replacing Bincode to improve performance and support newer transaction formats like V1 [1][2][3]. In sdk/src/transaction/versioned/mod.rs (or the corresponding solana-transaction crate), the VersionedTransaction struct implements serialization logic using Wincode traits [1]. Specifically: 1. Wincode Implementation: Wincode integration is guarded by the wincode feature flag [1]. The VersionedTransaction struct derives UninitBuilder and implements SchemaWrite and SchemaRead traits for Wincode serialization and deserialization [1]. 2. Serialization Mechanism: The implementation defines how signatures and message are serialized into the byte buffer [1]. For example, the signatures field is typically configured with a container like wincode(with = "containers::Vec<Signature, ShortU16>") [1]. 3. Versioning: The read method for VersionedTransaction uses a discriminator check (typically involving a message version prefix) to determine the transaction format (e.g., Legacy vs. V0 vs. V1) [1]. The transition to Wincode was motivated by the fact that Bincode is no longer maintained and does not natively support the V1 transaction format, which increases maximum transaction size and embeds compute budget configuration [2][3]. As of August 2026, Wincode is the standard approach for this component in the Solana SDK [1][2][3].

Citations:


🌐 Web query:

github solana-labs solana sdk/src/transaction/versioned/mod.rs VersionedTransaction Serialize wincode

💡 Result:

The VersionedTransaction struct in the Solana SDK uses the wincode crate for serialization and deserialization, specifically when the wincode feature is enabled [1][2][3]. In the source code (solana-transaction/src/versioned/mod.rs), VersionedTransaction is implemented with the wincode attribute and traits to support this serialization format [2][4]. Key technical details include: 1. Feature Flag: The wincode-related implementations, such as SchemaRead and SchemaWrite, are guarded by the wincode feature flag [1][2][3]. 2. Custom Serialization: The struct uses custom serialization attributes for its fields, for example: #[cfg_attr(feature = "wincode", wincode(with = "containers::Vec<Signature, ShortU16>"))] pub signatures: Vec, This ensures that the signatures field is serialized with specific container and length-prefix logic compatible with the Solana protocol [2]. 3. Traits: VersionedTransaction implements SchemaRead and SchemaWrite from the wincode crate, allowing it to be serialized into or deserialized from byte buffers [2][3]. 4. Evolution: Support for serialization formats (including updates to wincode versions and UninitBuilder patterns) is actively managed to support transaction versions such as V0 and V1 [5][4]. For most developers interacting with VersionedTransaction, the standard approach remains using the serialization methods provided by the high-level Solana client libraries (e.g., @solana/web3.js), which handle the underlying serialization (often via bs58 or base64 encoding of the binary buffer) automatically [6][7][8]. The wincode-specific implementation is primarily relevant for low-level protocol operations or when working directly with the Rust SDK [1][2].

Citations:


🌐 Web query:

gh api solana-labs/solana/git/refs/tags v4.1.1 --jq '.object.sha'

💡 Result:

To retrieve the SHA of the v4.1.1 tag in the solana-labs/solana repository using the GitHub CLI (gh), you should use the following command: gh api repos/solana-labs/solana/git/ref/tags/v4.1.1 --jq '.object.sha' [1][2] Note the following details regarding this command: 1. Path Requirement: The endpoint must be structured as /repos/{owner}/{repo}/git/ref/{ref}, where the {ref} component for a tag is formatted as tags/TAG_NAME [2][3]. The path used in your query, /git/refs/tags, is often used to list all tags rather than fetch a specific one [4][3]. 2. Endpoint Behavior: The GitHub REST API "Get a reference" endpoint is the correct way to retrieve the SHA of a specific tag [2]. 3. Repository Context: While v4.1.1 exists in related repositories like anza-xyz/agave [5][6], please verify that you are targeting the intended repository (solana-labs/solana), as tag availability may vary across different forks and projects within the Solana ecosystem [5][6][7]. If the tag v4.1.1 exists in the specified repository, the command will output the corresponding commit SHA directly [1][2].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== github v4.1.1 ref tag sha =="
gh api "repos/solana-labs/solana/git/ref/tags/v4.1.1" --jq '{object_type: .object.type, sha: .object.sha}' || true

echo "== fetch sdk versioned mod.rs for v4.1.1 =="
url="$(gh api "repos/solana-labs/solana/git/ref/tags/v4.1.1" --jq '.object.type as $type | if $type == "tag" then "repos/solana-labs/solana/git/tags/$(.object.sha)" else "repos/solana-labs/solana/git/trees/$(.object.sha)" end')"
sha="$(gh api "$url" --jq '.object.sha')"
echo "tag object sha $sha"
gh api "repos/solana-labs/solana/contents/sdk/src/transaction/versioned/mod.rs?ref=$sha" --jq '.content' | base64 -d | sed -n '1,240p'

echo "== fetch solana-message v1 try_compile from same tag if available =="
file="$(gh api "repos/solana-labs/solana/git/trees/$sha" --jq -r '.tree[]? | select(.path=="/sdk/program/src/message/mod.rs") | .path')"
echo "$file"
gh api "repos/solana-labs/solana/contents/$file?ref=$sha" --jq '.content' | base64 -d | sed -n '1,240p'

Repository: magicblock-labs/magicblock-engine

Length of output: 411


🌐 Web query:

docs.rs solana-message 4.1.1 source message mod.rs v1::Message try_compile

💡 Result:

In the solana-message crate version 4.1.1, the try_compile method is not available on a generic v1::Message, as there is no v1::Message structure that serves as a primary, top-level API for creating transactions [1][2][3]. The solana-message crate primarily defines two message versions for transaction handling: legacy (the original format) and v0 (which supports address lookup tables) [1][2][3]. The v0::Message structure provides a try_compile function that is widely used for creating signable transaction messages with address lookup tables [4][5]. Recent development (as of early 2026) has introduced support for a V1 transaction format [6]. The try_compile functionality is implemented within specific versioned modules where applicable (such as v0) to facilitate the construction of messages from a payer, a set of instructions, recent blockhash, and optional address lookup table accounts [4][7][5]. If you are looking to construct a versioned transaction, you should target v0::Message::try_compile [4][5] or the appropriate version-specific module as defined by the latest SDK, as there is no v1::Message::try_compile API in version 4.1.1 [8].

Citations:


🌐 Web query:

github solana-labs solana refs/tags v4.1.1 sdk/src/transaction/versioned/mod.rs

💡 Result:

In the Solana codebase, sdk/src/transaction/versioned/mod.rs defines the VersionedTransaction struct [1][2]. This structure is designed to support transactions with multiple message versions (such as legacy transactions and versioned transactions utilizing Address Lookup Tables) [2]. The VersionedTransaction struct is defined as follows: pub struct VersionedTransaction { pub signatures: Vec, pub message: VersionedMessage, } Key functionality provided by the implementation in this module includes: - sanitize: Validates the transaction, including checking signature counts and ensuring they match the message requirements [1][2]. - version: Returns the version of the transaction [1][2]. - into_legacy_transaction: Attempts to convert the transaction back into a legacy format if the underlying message is legacy [1][2]. - verify_and_hash_message: Verifies the signatures and hashes the message [1][2]. - uses_durable_nonce: Checks if the transaction begins with an advance nonce instruction [2][3]. While there are various v4.1.1 releases across different repositories (such as anza-xyz/agave [4] and solana-transaction [5]), the core structure of VersionedTransaction remains consistent across the Solana SDK versions, serving as the standard wrapper for signed transactions and their corresponding messages [6][2].

Citations:


🌐 Web query:

github solana-labs solana refs/tags v4.1.1 sdk/program/src/message/mod.rs

💡 Result:

In the Solana ecosystem, versioning for the core repository (solana-labs/solana) and individual crates like solana-message are distinct [1][2][3]. While the main solana-labs/solana repository reached version v1.18.x before being archived [1][2], the solana-message crate uses a different versioning scheme and reached version 4.1.1 [3][4]. The file sdk/program/src/message/mod.rs in the Solana SDK serves as the primary entry point for transaction message handling [5][6][4]. As of the solana-message v4.1.1 release, the mod.rs file establishes the architecture for transaction messages [4]: 1. Core Modules: It defines two primary versions of a Message: legacy and v0 [5][6][4]. 2. Functionality: The legacy format is the standard for backwards compatibility [5][6][4]. The v0 format enables more efficient encoding, allowing for a greater number of account keys compared to the legacy format [5][6][4]. 3. Architecture: The VersionedMessage type acts as a wrapper that supports both message versions [5][6][4]. 4. Program Limitations: Despite being located in the solana-program crate, the module notes that runtime messages cannot be directly accessed from within a Solana program; only the legacy message types are exposed for compatibility reasons [5][6][4]. You can view the documentation for this specific crate version on Docs.rs [4]. The solana-labs/solana repository itself has been archived, with ongoing development for newer versions transitioning to the Agave client [1].

Citations:


🌐 Web query:

github anza-xyz agave releases v4.1.1

💡 Result:

Agave version v4.1.1 was released on July 2, 2026 [1]. The release included a backported fix for a simpler nonce-size check for block production (pull request #13505) [1]. It was preceded by v4.1.0 (released June 26, 2026) [2] and followed by v4.1.2 (released July 10, 2026) [3]. The version bump to 4.1.1 was finalized in commit 19e19df [4].

Citations:


🌐 Web query:

github anza-xyz agave sdk/program/src/message/mod.rs v1 try_compile

💡 Result:

In the Agave repository (anza-xyz/agave), there is no function named try_compile located within the sdk/program/src/message/mod.rs file [1]. The message module primarily handles transaction message structures, including re-exporting legacy messages and managing compiled keys [1]. Compilation logic for transaction messages in the Solana SDK (which Agave maintains) is typically handled by the CompiledKeys::compile method [2], which is used during the construction of a Message [2]. It is possible that the term try_compile refers to a different utility, a custom implementation, or a misremembered function name, as the codebase utilizes standard Rust idioms like Result types and standard compilation patterns [1][2]. The message-related logic has undergone significant restructuring in recent versions, such as being moved to its own crate to improve modularity [3].

Citations:


🌐 Web query:

github anza-xyz agave sdk/src/transaction/versioned/mod.rs wincode

💡 Result:

Wincode is a high-performance, bincode-compatible serialization and deserialization library developed by Anza for the Agave validator [1][2][3]. It is designed to minimize CPU cycles and intermediate memory buffering by supporting in-place initialization and direct memory writes [1][3][4]. In the context of the Agave repository (including paths like sdk/src/transaction/versioned/mod.rs), wincode has been systematically replacing bincode for performance-critical serialization paths [5][2][4]. Key aspects include: - Compatibility: Wincode is bit-for-bit compatible with the bincode wire format, allowing it to serve as a drop-in replacement for existing Solana data structures [1][2][3]. - Technical Goals: Unlike traditional serializers that may require intermediate staging buffers, wincode operates using traits (such as SchemaRead and SchemaWrite) that facilitate direct memory access and in-place initialization [1][3][6]. - Implementation: Developers use derive macros (SchemaRead/SchemaWrite) and sometimes #[repr(C)] annotations to enable wincode support on data structures [7][8][3]. - Adoption: Its usage has been expanding across the Agave codebase, including the ledger, transaction processing, and snapshot serialization, as part of performance optimization efforts for releases such as Agave 4.0 [7][8][2][4].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== fetch v4.1.1 tree for transaction/message files =="
git ls-files -s | rg '(^|/)Cargo\.toml$|engine/src/transaction\.rs$' || true
tag_sha="$(git ls-remote --tags origin refs/tags/v4.1.1 | tail -n1 | cut -f1 || (git ls-remote origin && false))"
echo "v4.1.1 tag ref sha $tag_sha"
files="$(git ls-tree -r "$tag_sha" --name-only | grep -E '^sdk/(?:(?P<a>program/src/message/.*\.rs)|(?:src/transaction/versioned/.*\.rs))$' || true)"
echo "$files"

while IFS= read -r file; do
  [ -n "$file" ] || continue
  echo "--- $file ---"
  git show "$tag_sha:$file" | sed -n '1,220p'
done

Repository: magicblock-labs/magicblock-engine

Length of output: 2219


🌐 Web query:

github anza-xyz solana-sdk msg@4.1.1 sdk/src/transaction/versioned/mod.rs

💡 Result:

The path sdk/src/transaction/versioned/mod.rs was a standard location for the VersionedTransaction struct in earlier versions of the Solana SDK (historically maintained under solana-labs/solana) [1]. Following the transition to the Agave validator client organization (anza-xyz), the project structure has undergone significant refactoring, including the decoupling of core crates [2][3][4]. In the anza-xyz/solana-sdk repository, VersionedTransaction is typically found within the transaction crate, often located at transaction/src/versioned/mod.rs [5][6][7]. The implementation defines an atomic transaction containing a list of signatures and a VersionedMessage [1]. Recent developments in the anza-xyz/solana-sdk repository have introduced support for V1 transactions, which aim to increase the maximum transaction size and embed compute budget configuration directly into the message header [8][9]. This work has involved adding new variants to VersionedMessage and updating the associated transaction serialization and validation logic in the transaction and message crates [8][9]. For the specific version msg@4.1.1, please note that solana-sdk and its sub-crates often use independent versioning [10][7]. You can verify the exact file content and structure by browsing the corresponding tag or commit hash directly on the anza-xyz/solana-sdk GitHub repository [2].

Citations:


🌐 Web query:

github anza-xyz solana-sdk msg@4.1.1 sdk/program/src/message/mod.rs try_compile

💡 Result:

In the anza-xyz/solana-sdk repository, specifically within the version of the crate associated with message 4.1.1 [1][2], the try_compile function is a method implemented for the v0::Message struct [3][4]. This function is located within the solana-message crate (often found in the message directory of the repository) at src/versions/v0/mod.rs [3]. It is used to create a signable Version 0 transaction message [3][4]. The signature for this function is [3][4]: pub fn try_compile( payer: &Address, instructions: &[Instruction], address_lookup_table_accounts: &[AddressLookupTableAccount], recent_blockhash: Hash) -> Result<Self, CompileError> It takes a payer's public key, a slice of instructions, a slice of address lookup table accounts, and a recent blockhash to return a compiled v0::Message or a CompileError [3][4]. This pattern is commonly used when constructing VersionedTransaction objects that require address lookup tables for extended account access [4][5].

Citations:


🌐 Web query:

github anza-xyz solana-sdk msg@4.1.1 src versions v1 mod.rs try_compile

💡 Result:

The codebase under anza-xyz/solana-sdk contains a solana-message crate (often versioned separately, e.g., v4.1.1) which implements support for versioned transactions, including a v1 message format [1][2]. In the repository structure for solana-message (often located within the message/ directory of the solana-sdk repo), the path message/src/versions/v1/mod.rs exists as part of the implementation of V1 transactions [1][3]. Regarding try_compile: 1. The term try_compile appears in the context of helper functions added during the development of V1 transaction support (specifically in pull request #538, where commit 698b0a0 is noted as "Add try_compile helpers") [1]. 2. These helpers were introduced to facilitate the construction or validation of the new V1 message format, which supports larger transaction sizes (up to 4096 bytes) and embedded compute budget configurations [1]. 3. While the specific function signature or implementation might evolve, these utilities are designed to handle the complexities of the V1 message structure during the compilation or serialization process [1][4]. You can inspect the exact implementation by navigating to the relevant file in the anza-xyz/solana-sdk repository on GitHub, specifically within the message/src/versions/v1/ directory [1][4]. The repository documentation and pull request history (e.g., PR #538 and PR #622) provide context on why these try_compile and serialization helpers were necessary for the transition to the V1 message format [1][4].

Citations:


🌐 Web query:

github anza-xyz solana-sdk msg@4.1.1 src/versioned/mod.rs VersionedTransaction wincode signatures

💡 Result:

In the anza-xyz/solana-sdk repository, the VersionedTransaction struct—found in src/versioned/mod.rs—includes a signatures field defined as a vector of signatures (pub signatures: Vec) [1][2]. The use of wincode within the context of VersionedTransaction relates to the serialization and deserialization of these transactions, where wincode has been adopted to replace bincode in recent versions of the SDK [3]. Key details regarding this implementation include: 1. Wincode Integration: The SDK has transitioned to using wincode for transaction serialization, which involved updating the SchemaRead implementation for VersionedTransaction [4][3]. This update included moving from older derive attributes to newer mechanisms like UninitBuilder to handle memory safety and custom serialization requirements [4]. 2. Signature Handling: The signatures field is a compact-encoded array [5]. When using features enabled by wincode, methods such as try_new are available to sign versioned messages and generate the corresponding VersionedTransaction [1]. 3. Evolution of Implementation: The repository has seen continuous updates to its wincode usage, including modifications to SchemaReadContext and the removal of legacy reader methods (like peek_byte) to maintain compatibility with newer versions of the wincode crate [6]. For technical implementation details, you can reference the VersionedTransaction struct definition and its associated wincode trait implementations directly in the source file src/versioned/mod.rs within the repository [1][4][6].

Citations:


🌐 Web query:

github anza-xyz solana-sdk msg@4.1.1 try_compile num_required_signatures

💡 Result:

In the context of the anza-xyz/solana-sdk repository, particularly around the solana-message crate version 4.1.1, num_required_signatures is a field within the MessageHeader struct [1][2]. It represents the total number of required signatures for a transaction, which must match the first num_required_signatures of the account_keys in the transaction message [1][2][3]. Regarding try_compile, recent developments in the repository (such as in Pull Request #538) introduced new message formats and helper functions, including try_compile helpers, to support versioned transactions (e.g., V1 transactions) [4]. While specific implementation details of try_compile can evolve, these helpers are typically used to construct transaction messages by correctly setting header fields, including num_required_signatures, based on the provided instructions and account metadata [4][5]. There is no direct, singular public function named try_compile_num_required_signatures; rather, try_compile is often part of a broader set of builders or helpers intended to correctly calculate and validate the MessageHeader fields [4]. If you are encountering an issue related to this, it is likely related to how these builders verify that the number of required signatures is consistent with the provided account keys and instructions, or a check to prevent overflow or out-of-bounds errors when these fields are processed [6][5][7].

Citations:


Reject instruction lists that require an external signer.

v1::Message::try_compile counts signers in message.header.num_required_signatures. If an instruction requires a second signer, allocate multiple signature slots and sign them before replacing the Magicblock prefix; allocating one slot while the header requires more emits malformed Magicblock transaction data.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/transaction.rs` around lines 90 - 121, Update magicblock to reject
compiled messages whose header.num_required_signatures exceeds the Engine
authority’s single-signature model. Validate this immediately after
v1::Message::try_compile and return an appropriate transaction error before
constructing VersionedTransaction, ensuring external-signer instructions cannot
produce malformed data.

Comment thread engine/tests/recovery.rs
Comment on lines +58 to +59
// Guarded by a timeout so a replay regression fails fast instead of hanging.
let te2 = TestEngine::with(dirs, authority).await;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

The comment claims a timeout that the code does not apply.

Line 58 states that the reopen is guarded by a timeout. Line 59 calls TestEngine::with(dirs, authority).await directly, with no time::timeout wrapper. A replay regression that hangs will hang this test instead of failing fast. replay_aborts_on_checksum_mismatch applies the guard correctly at Lines 90-95.

Add the timeout, or remove the comment.

🧪 Proposed fix: apply the timeout the comment describes
     // Guarded by a timeout so a replay regression fails fast instead of hanging.
-    let te2 = TestEngine::with(dirs, authority).await;
+    let te2 = time::timeout(Duration::from_secs(4), TestEngine::with(dirs, authority))
+        .await
+        .expect("replay completes in time");
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Guarded by a timeout so a replay regression fails fast instead of hanging.
let te2 = TestEngine::with(dirs, authority).await;
// Guarded by a timeout so a replay regression fails fast instead of hanging.
let te2 = time::timeout(Duration::from_secs(4), TestEngine::with(dirs, authority))
.await
.expect("replay completes in time");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/tests/recovery.rs` around lines 58 - 59, Wrap the
TestEngine::with(dirs, authority).await call in the
replay_aborts_on_checksum_mismatch test with the same timeout mechanism used at
Lines 90-95, preserving the existing reopen behavior while ensuring hangs fail
fast. Keep the timeout comment aligned with the implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add top-level engine orchestration crate

1 participant