Using a Computing Cluster

For DNA origamis larger than e.g. 1000nt scaffold strand, the scaffoldselector can become computationally heavy due to the large number of sequence comparisons necessary. On a single machine, sometimes several days of compute time are necessary.

To reduce the computation time, the program can be run on a computing cluster where it assigns one computing core per scaffold sequence (or scaffold strand rotation). This enables many scaffold sequences to be scored independently and in parallel, significantly speeding up the scoring of a large pool of scaffold sequences.

The scaffoldselector code is compatible with computing clusters implementing a SLURM workload manager. The cluster must support running Python scripts and the creation of Python virtual environments.

In our Paper [ref], we used a SLURM computing cluster to find optimal scaffold sequences for a DNA rectangle (2484nt scaffold) and DNA triangle (2410nt scaffold), by selecting scaffolds as regions of 5 biological vectors. In the below examples, we detail how to reproduce our results.

Install Software on Cluster

Log into your computing cluster and follow the Installation instructions for the scaffoldselector software (the same instructions as if you are installing on a local machine). The NUPACK folder can be moved to the cluster using e.g. an SFTP client. Make sure you use the same version of NUPACK on the cluster as on your local machine.

Worked Example: DNA Origami Rectangle

The DNA origami rectangle files are below:

  • Rectangle contact map: rect.csv (Right click, Save As)

  • Rectangle scaffold pool file of 5000 scaffolds: rect_pool.csv (Right click, Save As). This scaffold pool file was created by randomly sampling regions 2484nt regions from biological vectors p7249, p7560, p8064, pUC19 and lambda DNA.

Once the software is installed on the cluster, select the optimal sequences for the DNA origami rectangle from the scaffold pool by following these instructions:

  1. Log into your cluster, activate Python and change to the scaffold selector directory:

    ssh <your cluster>
module load Python
cd origami
source venv/bin/activate
cd scaffoldselector

  2. Upload the rect.csv contact map file and the rect_pool.csv scaffold pool file to the scaffoldselector/_input directory.

  3. Open the settings file:

    nano settings.json

  4. Make the settings read as below, then Ctrl-X y Enter to exit:

    {
        "mode 1 metrics"             : "1,3,4",
        "mode 2 metrics"             : "1,2,3,4",

        "mode 2 scaffold pool file"  : "rect_pool.csv",
        "mode 2 rotate scaffold"     : "No",
        "mode 2 number of samples"   : 5000,

        "scaffold material"          : "DNA",
        "staple material"            : "DNA"
}

  1. Submit the whole job to the cluster by typing:

    python3 selector.py rect mode2 -cluster

Instead of scoring all origamis locally, the scaffold selector creates the arrayjob script scaffoldselector/arrayjob_rect0.sh below:

#!/bin/bash
#
# set up a job array with tasks numbered 0 to total_jobs-1 (i.e. total_jobs in total)
#SBATCH --array=0-999
#
# give the array a single job name
#SBATCH -J rect
#
# Standard output is saved in this file. A is the jobid and a is the array index (A and a are added by SLURM)
#SBATCH -o _output/rect/logs0/%A_%a.out
#
# Standard error messages are saved in this file. A is the jobid and a is the array index (A and a are added by SLURM)
#SBATCH -e _output/rect/logs0/%A_%a.err
#
# request one core per task
#SBATCH -c 1
#
# request run-time for this WHOLE array job
#SBATCH -t 0-10:00:00
#
# Tell SLURM if you want to be emailed when your job starts, ends, etc.
#SBATCH --mail-type=ALL
#
# use the task ID to locate the input file for the task.
python3 selector.py rect score 0 ${SLURM_ARRAY_TASK_ID}

Then, the scaffold selector submits the arrayjob to the cluster using command sbatch arrayjob_rect0.sh.

  1. Check the status of all jobs using:

    squeue -u <your cluster username>

  2. When the above job finishes, if other array job scripts exist, then execute each of these manually too. For example, execute sbatch arrayjob_rect1.sh, again wait for jobs to finish, then execute sbatch arrayjob_rect2.sh and so on, until all array jobs have been done. This gets around the queue size limit that many computing clusters have (see Technical Notes below).

  3. When all jobs have completed on the computing cluster, compress the results into a zipped tarball file by submitting one final job to the cluster:

    sbatch compress_rect.sh

  4. When the above job completes, download file scaffoldselector/_output/rect.tar.gz to your local machine in the same location in the scaffold selector repo.

  5. Run the following commands on your local machine, to unzip the results and process them:

    cd origami
source venv/bin/activate
cd scaffoldselector
python3 selector.py rect collate
python3 selector.py rect results

  6. The HTML output report produced at scaffoldselector/_results/rect.html should look as follows: rect.zip (Right click, Save As)

Worked Example: DNA Origami Triangle

The DNA origami triangle files are below:

  • Triangle contact map: tri.csv (Right click, Save As)

  • Triangle scaffold pool file of 5000 scaffolds: tri_pool.csv (Right click, Save As). This scaffold pool file was created by randomly sampling regions 2410nt regions from biological vectors p7249, p7560, p8064, pUC19 and lambda DNA.

To score all scaffolds, follow the same procedure above as for the rectangle.

The HTML output report produced at scaffoldselector/_results/tri.html should look as follows: tri.zip (Right click, Save As)

Technical Notes

  1. Many computing clusters have a queue size limit, i.e. the maximum number of jobs you can submit in one go. The queue size limit is set as CLUSTER_MAX_JOBS_IN_QUEUE in constants.py. Change this number to suit your computing cluster. Some clusters have 1000 limit, others have 10000 limit. If the number of jobs sent (in this case 5000) exceeds the cluster queue size, then the arrayjob is split into smaller array jobs, e.g. arrayjob_rect0.sh, arrayjob_rect1.sh and so on. You have to manually sbatch arrayjob_rect1.sh when arrayjob_rect0.sh finishes.

  2. All code for submitting cluster jobs is in the scaffoldselector/cluster.py file. If your computing cluster is not SLURM-based, or uses different commands, you can edit this file and e.g. change the arrayjob file format and replace the sbatch command with whatever submit function your cluster uses for arrayjobs. The main requirement is that your computing cluster supports Python and allows installation of Python libraries.