Submit estimate gene expression job to SLURM cluster

Source: R/estimate_expression.R

Reads gene expression estimate jobs from

submit_estimate_expression_slurm(
  tag,
  base_dir,
  scratch_dir,
  slurm_account,
  slurm_mail_user,
  slurm_partition
)

Arguments

tag

label for scratch and result directories
base_dir

directory for project
scratch_dir

scratch directory for the project
slurm_account

slurm account
slurm_mail_user

email for slurm job updates at BEGIN,END
slurm_partition

partition on slurm cluster where to run the job

Details

intermediate_data/todo_runs_<tag>.tsv # tab separated table with column [run_accession]

outputs results to

intermediate_data/estimate_expression_<tag>

This submission script executes

cd <base_dir> sbatch \ --array 1-<n_runs> \ --export=TAG='<ta>',BASE_DIR='<base_dir>',JOB_DIR='<job_dir>' \ ~/opt/CalCEN/inst/run_estimate_expression_SLURM_wrapper.sh . <tag>

A demon of running from the shell

export BASE_DIR=/home/maom/opt/CalCEN export JOB_DIR=/scratch/maom_root/maom99/maom/CalCEN/estimate_expression_20201007 export SLURM_ARRAY_TASK_ID=1 ~/opt/CalCEN/inst/run_estimate_expression_SLURM_wrapper.sh

this will set:

TAG: 20201007 BASE_DIR: /home/maom/opt/CalCEN/vignettes/<network> SLURM_ARRAY_TASK_ID: 1 Rscript: /home/maom/opt/bin/Rscript RUNS_FNAME: /home/maom/opt/CalCEN/intermediate_data/todo_runs_20201007.tsv LOGS_DIR: /home/maom/opt/CalCEN/intermediate_data/estimated_expression_20201007/logs TASK_NAME: estimated_expression_1 JOB_DIR: /scratch/maom_root/maom99/maom/CalCEN/estimated_expression_20201007 TASK_DIR: /scratch/maom_root/maom99/maom/CalCEN/estimated_expression_20201007/estimated_expression_1

and then call

Rscript ~/opt/CalCEN/scripts/run_estimate_expression.R \ --runs_fname $BASE_DIRintermediate_data/todo_runs_20201007.tsv \ --run_id 1 \ --results_dir $BASE_DIR/intermediate_data/estimated_expression_20201007 \ --logs_dir $BASE_DIR/intermediate_data/estimated_expression_20201007/logs \ --work_dir $TASK_DIR/estimated_expression_1