IBG Wiki

User Tools

Site Tools

Trace:
keller_and_evans_lab:cu_research_computing

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
keller_and_evans_lab:cu_research_computing [2016/04/07 15:25]
lessem /* Don't save temporary files in /work/KellerLab */ 		keller_and_evans_lab:cu_research_computing [2016/12/08 15:16]
matthew_keller /* Getting information on jobs */
Line 59: Line 59:
  
  
-You will have to manually create a directory to put their stuff in. You can also just make a big mess with files all over and annoy other users. lustre and rc_scratch are network filesystems, and will appear identical on all nodes that connect to them. /local/scratch is local to the particular node used, so something saved on bnode0108 will not be visible from himem04. The size of /local/scratch depends on which node is used, but it is about not large. The "df" command does not work on rc_scratch, so it's unclear how much space is available. In the future lustre will be going away, and will probably be replaced by having rc_scratch.+You will have to manually create a directory to put their stuff in. You can also just make a big mess with files all over and annoy other users. lustre and rc_scratch are network filesystems, and will appear identical on all nodes that connect to them. /local/scratch is local to the particular node used, so something saved on bnode0108 will not be visible from himem04. The size of /local/scratch depends on which node is used, but it is about not large. The "df" command does not work on rc_scratch, so it's unclear how much space is available. In the future lustre will be going away, and will probably be replaced by rc_scratch.
  
  
 ======= Slurm ======= ======= Slurm =======
 +
 +
 +
 +====== Queues ======
 +
 +
 +#if you want to run on ibg himem, you need to load the right module
 +module load slurm/blanca
 +
 +#then in your shell script
 +#SBATCH --qos=blanca-ibg
 +
 +#If you want to run on normal queues, then:
 +module load slurm/slurm
 +
 +#then in your shell script, one of the below, depending on what queue you want
 +#SBATCH --qos=himem
 +#SBATCH --qos=crestone
 +#SBATCH --qos=janus
 +
 +
  
  
Line 68: Line 89:
  
  
-squeue -u <username>+ 
 +#To check our balance on our allocations and get the account id# 
 +sbank balance statement 
 +sacctmgr -p show user <username> #alternatively to find the acct# 
 + 
 +#To see how busy the nodes are. For seeing how many janus nodes are available, look for the  
 +#number under NODES where STATE is "idle" for PARTITION "janus" and TIMELIMIT 1-00:00:00. 
 +sinfo -l   
 + 
 +#checking on submissions for a user 
 +squeue -u <username>  #To see your job statuses (R is for running, PD pending, CG completing, CD completed, F failed, TO timeout)
 squeue -u <username> -t RUNNING squeue -u <username> -t RUNNING
 squeue -u <username> -t PENDING squeue -u <username> -t PENDING
 +squeue -u <username> --start #Get an estimate of when jobs will start
 +
 +#detailed information on a queue (who is running on it, how many cpus requested, memory requested, time information, etc.)
 +squeue -q blanca-ibg -o %u,%c,%e,%m,%j,%l,%L,%o,%R,%t
 +
 +#current status of queues
 +qstat -i #To see jobs that are currently pending (this is helpful for seeing if queue is overbooked)
 +qstat -r #To see jobs that are currently running
 +qstat -a #To see jobs that are running OR are queued
 +qstat -a -n #To see all jobs, including which nodes they are running on
 +qstat -r -n #To see running jobs, and which nodes they are running on
 +
 +#other commands
 showq-slurm -o -U -q <partition>  #List job priority order for current user (you) in given partition showq-slurm -o -U -q <partition>  #List job priority order for current user (you) in given partition
-scontrol show jobid -dd <jobid>   #List detailed information for a job (useful for troubleshooting)+scontrol show jobid -dd <jobid>   #List detailed information for a job (useful for troubleshooting). More info [https://www.rc.colorado.edu/book/export/html/613 here]. 
 +pbsnodes -a #To look at the status of each node
  
 ### Once job has completed, you can get additional information  ### Once job has completed, you can get additional information 
Line 80: Line 125:
 sacct -u <username> --format=JobID,JobName,MaxRSS,Elapsed  #View same info for all jobs of user sacct -u <username> --format=JobID,JobName,MaxRSS,Elapsed  #View same info for all jobs of user
  
 +#To check graphically how much storage is being taken up in /work/KellerLab folder 
 +xdiskusage /work/KellerLab/sizes
  
  
-====== Controlling jobs ====== 
  
  
 +====== Running and Controlling jobs ======
 +
 +
 +sbatch <shell.script.name.sh> #run shell script
 +sinteractive --nodelist=bnode0102 #run interactive job on node "bnode0102"
 scancel <jobid>                  #Cancel one job scancel <jobid>                  #Cancel one job
 scancel -u <username>            #Cancel all jobs for user scancel -u <username>            #Cancel all jobs for user
Line 113: Line 164:
 This only needs to be done once. This only needs to be done once.
  
-Then launch your interactive job.+Then launch your interactive job on the IBG himem node.
  
 module load slurm/blanca && sinteractive --qos=blanca-ibg module load slurm/blanca && sinteractive --qos=blanca-ibg
 +
 +
 +Or onto any free himem node
 +
 +module load slurm/blanca && sinteractive --qos=blanca
  
  
Line 180: Line 236:
  
 tabix -h chr${chr}/chr${chr}impv1.vcf.gz ${chr}:${startpos}-${endpos} | bgzip -c > chr$chr/chr${chr}impv1.${chr}_$startpos-$endpos.vcf.gz tabix -h chr${chr}/chr${chr}impv1.vcf.gz ${chr}:${startpos}-${endpos} | bgzip -c > chr$chr/chr${chr}impv1.${chr}_$startpos-$endpos.vcf.gz
 +
 +
 +
 +======= Compiling software =======
 +
 +RC intentionally keeps some header files off the login nodes to dissuage people from trying to compile on those nodes. Instead, use the janus-compile nodes to compile your software. Log in to a login node and then run
 +
 +ssh janus-compile[1-4]
  
  
keller_and_evans_lab/cu_research_computing.txt · Last modified: 2020/02/12 09:03 by lessem

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
CC Attribution-Noncommercial-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki