Overview

Benchpress is a Snakemake workflow based on a collection of packages, libraries, and open-source software related to structure learning for graphical models. It is designed in a modular way, separating graph (Graphs), parameterization (Parameters), and data sampling (Data), from structure learning (Algorithms) and performance evaluation (Evaluation), so that it “..can be seamlessly scaled to server, cluster, grid and cloud environments, without the need to modify the workflow definition.”, following the Snakemake principles. In order to maintain reproducibility and to save the user from manually installing each of the used software, Benchpress relies on Snakemake’s support for Apptainer and runs by default each of the modules in separate containers.

Installation

The first step is to install git and to clone the Benchpress repository to a directory of choice (path/to/benchpress)

git clone https://github.com/felixleopoldo/benchpress.git path/to/benchpress

To keep you up-to-date with the latest updates you simply do the git pull command in the benchpress directory.

As soon as Apptainer and Snakemake are available, no further installation is required and you are ready to run Benchpress. Here we describe two ways to get these in place.

Docker (Linux/macOS/Win)

For this alternative, you only need to install Docker, following the instructions for your specific system. This is usually the easiest way to get started and it works for any platform supporting Docker. You may configure the resources (memory, cores, disk,… ) to be used through the Docker interface.

Without Docker (Linux)

This alternative is preferred e.g. if you will add new modules or if you are working on a server that doesn’t support Docker (typically for security reasons).

Install Anaconda (or miniconda) from here.
Install Snakemake >= 7.30.1 following the installation instructions.
Install Apptainer (former Singularity) following the installation instructions.

Note

If Snakemake was installed with Mamba (as suggested on Snakemakes’ homepage) you may update it to the latest version by running

mamba update snakemake

Note

Some systems require explicit installation of squashfs-tools, which can be installed by conda as

conda install -c conda-forge squashfs-tools

Note

For cloud or cluster execution however, software such as Kubernetes or Slurm may also be required. See the Snakemake documentation for further information.

Usage

Here we describe some basic commands to get you started using Benchpress. Depending on if you use Docker or if you have installed Snakemake and Apptainer natively, follow one of the alternatives below.

Docker

Start an interactive Docker container (based on the official Snakemake docker image) in privileged mode, where the benchpress folder (absolute path) is shared as a volume, here to the folder /mnt which is also set to the working directory.

docker run -it -w /mnt --privileged -v /absolute/path/to/benchpress:/mnt bpimages/snakemake:v7.32.3

Depending on your system, you may have to prepend the above command by sudo. On Windows, you may have to escape spaces in the benchpress path using ^ or `.

Linux without Docker

If you installed Snakemake by using Conda as suggested by Snakemake, you first need to activate the conda environment called snakemake. First head into the benchpress folder by

cd path/to/benchpress

then run

conda activate snakemake

Run config file

Benchpress is designed to handle all the user interaction through a JSON config file, on a specific format that is described in JSON config. The following command runs the config file config/config.json, whether you use Docker or run natively on your machine, using all available cores

snakemake --cores all --use-singularity --configfile config/config.json

The output is stored in subfolders of path/to/benchpress/results/output named by the evaluation modules. Note that the first run will take some longer time, as about 5-10 GB of Docker images will be downloaded.

To use other modules, you may copy the JSON snippets from the documentation (Graphs, Parameters, Data, Algorithms, Evaluation) and paste them into your config file. It may also be helpful to look into the other config files in config/.

Note

For cluster and cloud execution, please review the Snakemake documentation for the commands to use.