Running Qflex

Running QFlex

In this case, QFlex is meant to be running in a virtual cluster on the same host. For this reason, the first step is to build QFlex with appropriate features, then to build a virtual network inside the host machine, to finally add the guest instance to the virtual network.

Requirements:

• Running a Linux distribution
• iproute2 is installed
• Being inside the Nix environment

Building QFlex using NixOS

NixOS is a platform that allows reproducible builds and deployments such as you won't need to manage any dependencies for running QFlex.

Please first clone the qflex main respository (at the moment all branches of qflex, libqflex, flexus, qemu need to checkout to qflex-8.1):

git clone https://github.com/parsa-epfl/qflex.git
cd qflex
git submodule update --init

To install nix, and enable Nix Flakes (~1 minute):

sh <(curl -L https://nixos.org/nix/install) --daemon # install nix
Welcome to the Multi-User Nix Installation
...
Would you like to see a more detailed list of what I will do?
[y/n] n
...
Can I use sudo?
[y/n] y
... nixbldXX users ...
Ready to continue?
[y/n] y
...
Alright! We're done!

Finally, to enable Nix Flakes:

mkdir -p ~/.config/nix
echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf

Finally, you can enter the nix environment and start building QEMU:

nix develop -i -k HOME # Entered the enviroment
cd qemu 
./configure --target-list=aarch64-softmmu --enable-qflex --enable-snapext --enable-capstone --enable-multinode
make -j12

And to build both QFlex Timing and Trace simulators:

cd ../libqflex/qflex-trace
make
cd ../../
cd flexus
mkdir build && cd build
cmake -DSIMULATOR=KeenKraken .. && make -j12
cmake -DSIMULATOR=KnottyKraken.Boom .. && make -j12 

Always make sure to be inside the nix environment running cmake or ./configure and building make -jX.

Notes:

• -i = --ignore-environment: Clear the entire environment (except those specified with --keep), it won't try to parse any local packages and libraries.
• -k NAME = --keep: Keep the environment variable name.
• --enable-snapext is for allowing QEMU to save incremental and external snapshots, check SMARTS methodology for more details.
• --enable-multinode is for allowing two or more QEMU instances to advance in sychronicity in the notion of time.
• --enable-qflex is for building QEMU modifications allowing to run with the simulator attached.
• --enable-capstone is for QEMU and QFlex to use capstone when logging executed instructions type.
• KeenKraken is our QFlex Trace simulator.
• KnottyKraken.Boom is our QFlex Timing simulator.

Running a QFlex simulation

Starting QEMU

Here we will start a QEMU instance with 1 CPU and 4GB of RAM.

QFLEX_ROOT=<Here get QFlex path>
HOST_SSH_PORT=2245
$QFLEX_ROOT/qemu/build/qemu-system-aarch64 --cpu max \
 -machine virt,gic-version=3 \
 -smp 1 -m 4G -rtc clock=vm \
 -drive file=$QFLEX_ROOT/images/qemu-efi.img,format=raw,if=pflash,readonly=on \
 -drive file=$QFLEX_ROOT/images/varstore.qcow2,if=pflash,readonly=on \
 -drive file=$QFLEX_ROOT/images/root.qcow2,format=qcow2,if=virtio \
 -device virtio-net-device,netdev=net1 \
 -netdev user,id=net1,hostfwd=tcp::$HOST_SSH_PORT-:22 \
 -icount shift=0,sleep=on,align=off \
 -snapvm-external \
 -nographic

Snapshot for Trace

Once your workload is in the middle of execution, you can interacting with QEMU in order to save a snapshot.

To access the QEMU prompt of any QFlex instance, press Ctrl-A then C. From the prompt, one key additional commands was implemented.

• savevm-external [NAME]: Save an incremental snapshot of the running machine

Loading a snapshot in Trace mode

Here we will run QEMU with the argumnets necessary to do trace based simulation.

Please change [NAME] to match previously saved snapshot:

QFLEX_ROOT=<Here get QFlex path>
HOST_SSH_PORT=2245
$QFLEX_ROOT/qemu/build/qemu-system-aarch64 --cpu max \
 -machine virt,gic-version=3 \
 -smp 1 -m 4G -rtc clock=vm \
 -drive file=$QFLEX_ROOT/images/qemu-efi.img,format=raw,if=pflash,readonly=on \
 -drive file=$QFLEX_ROOT/images/varstore.qcow2,if=pflash,readonly=on \
 -drive file=$QFLEX_ROOT/images/root.qcow2,format=qcow2,if=virtio \
 -device virtio-net-device,netdev=net1 \
 -netdev user,id=net1,hostfwd=tcp::$HOST_SSH_PORT-:22 \
 -qflex mode=trace,core-count=1,sim-path=$QFLEX_ROOT/flexus/build/libKeenKraken.so,sim-config-path=$QFLEX_ROOT/flexus/user_postload,debug=dev \
 -plugin $QFLEX_ROOT/libqflex/qflex-trace/libqflex-trace.so",
 -icount shift=0,sleep=on,align=off \
 -snapvm-external \
 -loadvm-external [NAME] \
 -nographic

Execute a few millions instructions and then save the uArch state for Timing simulation. You can save the snapshot identically to previous stage.

Run QFlex Timing

QFLEX_ROOT=<Here get QFlex path>
HOST_SSH_PORT=2245
$QFLEX_ROOT/qemu/build/qemu-system-aarch64 --cpu max \
 -machine virt,gic-version=3 \
 -smp 1 -m 4G -rtc clock=vm \
 -drive file=$QFLEX_ROOT/images/qemu-efi.img,format=raw,if=pflash,readonly=on \
 -drive file=$QFLEX_ROOT/images/varstore.qcow2,if=pflash,readonly=on \
 -drive file=$QFLEX_ROOT/images/root.qcow2,format=qcow2,if=virtio \
 -device virtio-net-device,netdev=net1 \
 -netdev user,id=net1,hostfwd=tcp::$HOST_SSH_PORT-:22 \
 -icount shift=0,sleep=on,align=off \
 -snapvm-external \
 -loadvm-external [NAME] \
 -qflex mode=timing,core-count=1,$QFLEX_ROOT/flexus/build/libKeenKraken.so,sim-config-path=$QFLEX_ROOT/flexus/user_postload,debug=dev \
 -d nochain -singlestep \
 -nographic

Once you have saved the snapshot