Difference between revisions of "Using Snakemake on the HPC Cluster"

Revision as of 19:50, 16 May 2024

Setup

• Install snakemake via mamba/conda (see https://snakemake.readthedocs.io/en/stable/getting_started/installation.html)
• In the working folder create a snakemake environment with

  mamba activate snakemake

• Create a generic profile in ~/.config/snakemake/testprofile/config.yaml such as:

executor: slurm
jobs: 100

default-resources:
    mem_mb: max(1.5 * input.size_mb, 100)
    account: <ACCOUNT_NAME>
    partition: <PARTITION NAME>
set-threads:
    myrule: max(input.size_mb / 5, 2)
set-resources:
    myrule:
        mem_mb: attempt * 200

• Create a Snakefile (see
  • You can over-ride resources with that flag otherwise it will use the values in testprofile
• Run the Snakefile with this command

  snakemake --profile testprofile  -j 1 --executor cluster-generic --cluster-generic-submit-cmd "batch" ...