Intro to bash scripting and HPC Workshop Setup instructions on REANNZ

Modified

August 19, 2026

GitHub : https://github.com/GenomicsAotearoa/Workshop-Bash_Scripting_And_HPC_Job_Scheduler

This workshop needs to be run on the REANNZ HPC cluster.

All learners and instructors will need to have an account on the REANNZ HPC cluster and will need to be added to the GA training project nesi02659 as team members before the workshop.

Dataset setup

Each user will get their own copy of the example dataset. These copies will be made in the nobackup dir, but the original dataset is stored in the project dir. The very first thing all learners and instructors will do in the workshop is create a symbolic link to their own copy of the dataset in their home directory. The benefit of this that:

  • Everyone can work on their own copy of the dataset without interfering with each other.
  • Everyone’s path to the dataset will be the same (~/scripting_workshop), so all workshop instructions will work for everyone.

The original dataset is stored at:

/nesi/project/nesi02659/scripting_workshop

which contains:

ls -F scripting_workshop/
modload.sh*  rna_seq/  scheduler/   variant_calling/

Instructions to setup dataset copies

There are instructions on setting up copies of the dataset in the project at:

/nesi/project/nesi02659/INSTRUCTOR_INSTRUCTIONS

These instructions below are the same as the instructions in the readme in the HPC/bash workshop instructions dir. This should be completed once by the lead training coordinator (right now that is Chloe) or a REANNZ staff member may assist and complete this. It is best to wait until you have all nesi user names from all participants (learners and instructors/helpers):

Steps:

  1. Change dir to nobackup and make a new scripting workshop dir
cd /nesi/nobackup/nesi02659/
mkdir scripting_workshop 
cd scripting_workshop
  1. Create a text file called .new and copy each user account IDs for each learner on a new line. Include all instructors and helpers too!
nano .new

Once you’ve entered the usernames and saved the .new file, it should look like:

cat .new
username1
username2
username3
  1. Copy the contents of the dataset to a hidden template folder
cp -r /nesi/project/nesi02659/scripting_workshop /nesi/nobackup/nesi02659/scripting_workshop/.template
ls -F /nesi/nobackup/nesi02659/scripting_workshop/.template
rnaseq/ scheduler/ variant_calling/ modload.sh
  1. Copy the script copy.sh to /nesi/nobackup/nesi02659/scripting_workshop/
cp /nesi/project/nesi02659/INSTRUCTOR_INSTRUCTIONS/intro-hpc-bash-script_WORKSHOP-INSTRUCTIONS/copy.sh .
  1. Run the copy.sh script. The output should make a new dir named for each user, and within each user dir all the scripting_workshop template files should be contained.
bash copy.sh

Expected output after successful copy.sh run:

ls -F /nesi/nobackup/nesi02659/scripting_workshop/users/*
/nesi/nobackup/nesi02659/scripting_workshop/users/username1:
modload.sh*  rna_seq/  scheduler/  variant_calling/

/nesi/nobackup/nesi02659/scripting_workshop/users/username2:
modload.sh*  rna_seq/  scheduler/  variant_calling/

/nesi/nobackup/nesi02659/scripting_workshop/users/username3:
modload.sh*  rna_seq/  scheduler/     variant_calling/
  1. Get each learner and instructor to create a symlink to their own dir above by running:
cd ~
ln -s /nesi/nobackup/nesi02659/scripting_workshop/users/${USER} scripting_workshop
  1. Good to go!
  2. At the end of workshop, run rm -rf /nesi/nobackup/nesi02659/scripting_workshop to remove all copies of the datasets.
Copy.sh 
cat copy.sh
#!/bin/bash -e
MAIN_DIR="/nesi/nobackup/nesi02659/scripting_workshop"
mkdir -p ${MAIN_DIR}/users
for ACCOUNT in $(cat ${MAIN_DIR}/.new)
do
    echo "setting up ${ACCOUNT} ..."
    cp -r ${MAIN_DIR}/.template ${MAIN_DIR}/users/${ACCOUNT}
    chmod -R 777 ${MAIN_DIR}/users/${ACCOUNT}
done

Backup copy

There is a back up of the dataset on GitHub:

https://github.com/GenomicsAotearoa/Workshop-Bash_Scripting_And_HPC_Job_Scheduler/releases/download/2026-April/scripting_workshop.zip