ObfusEval

ObfusEval is the benchmarking tool to evaluate the reliability of the code obfuscating transformation.

The following two metrics related the reliability are evaluated.

• Test Pass Rate indicates the rate of programs that are successfully obfuscated while keeping the functionalities.
• Code Distance Mean indicates how an obfuscated program is changed from the original program.

Requirement

Docker

• docker version 20.10.11 or newer
• docker-compose version 1.29.2 or newer

Rust

• cargo 1.64.0 or newer
• rustc 1.64.0 or newer
• rustup 1.25.1 or newer

Python (if plot result to graph using python script)

• python 3.9 or newer

How to use

Preparation

0. Clone this repository

git clone https://github.com/NAIST-SE/ObfusEval.git && cd ObfusEval

1. Add a docker image of an obfuscating transformation tool.

   • Please place the Dockerfile in docker/ subdirectory.
   • Define the obfuscate command in your Dockerfile as follows to handle commands related to obfuscation uniformly in this tool.

     RUN echo '#! /bin/sh\n<PATH TO THE OBFUSCATOR> $@' >> /bin/obfuscate && \
         chmod a+x /bin/obfuscate

2. Edit obfuscator/tool_name.json

   • Set the parameters as follows.
     • Please refer to the obfuscator existing file for details

     {
         "name": <TOOL_NAME>,
         "compiler": <PATH_TO_COMPILER_IN_TOOL>,
         "common_flag": <COMMON_FLAG_ON_COMPILING>, 
         "is_bin_only": <Whether the obuscator generates obfuscated binary only>,
         "transformations": [
             "name": <This transformation name>,
             "display": <This transformation abbreviation>,
             "flag": [
                 <Command line parametes to execute the docker image>
             ]
         ]
     }

3. Edit properties.json

   • Set the pair of obfuscator that contains obfuscating transformations and dataset.

     example

     {
         "name": "prop1",
         "dataset": "basic-algorithms", // the name of dataset/<each dataset>/
         "obfuscator": "tigress" // the name of obfuscator/<tool>.json
     }

4. Run docker service

   $ docker-compose up -d

Setup

• Apply obfuscation to the code in the dataset
• Compile code using compiler in docker service

$ cargo run setup <only_mode_option> <prop-name>

only_mode_option

• --compile-code: Compile code in dataset by compiler same using obfuscator
• --obfuscate-code: Obfuscate code in dataset

When run with no option, do all mode (compile and obfuscate code in dataset.)

Evaluate

$ cargo run evaluate <only_mode_option> <prop-name>

only_mode_option

• --test-pass-rate: Write discription
• --code-distance-mean: Write discription

When run with no option, do all mode.

LICENSE

• This source code in src is licensed MIT.

  Copyright (c) 2022 Kitaoka Tetsuya - Software Engineering Lab @ NAIST Released under the MIT license https://github.com/NAIST-SE/ObfusEval/blob/master/LICENSE

• The source code in dataset/basic-algorithms/src is derived from tum-i4/obfuscation-benchmarks.

  Copyright (c) 2015 Software Engineering Chair 22 - Faculty of Informatics Released under the MIT license https://github.com/tum-i4/obfuscation-benchmarks/blob/master/LICENSE