FNAL - KICP Joint Cluster

Fair Use Policy

• Currently, the shares of KICP and FNAL in the cluster are 30% and 70%. In order to prevent any single user from abusing the unrestricted access to the cluster, it is requested that KICP users limit their regular use to at most 150 cores, and FNAL users to 350 cores.
• Large single jobs (in excess of the above limits) are not forbidden. If you plan to use more than your limit of the cluster, please warn other users by e-mailing to cosmos-users@fnal.gov. Again, you are allowed to violate the limit for a single large job only.
• Serial (single-core) jobs that take all the memory on the node prevent other users from accessing the remaining 7 cores on that node. They, in effect, use 8 times more resources than they reserve, and waste 7/8 of them. Such jobs may be needed and are not forbidden, but, please, do not run more than 4 of them at any given time.
• The head node is the only entry point into the cluster. It has only 16GB of memory and 8 cores, so please refrain from running any code on it besides usual shell commands and compilers. The only exception to this rule is when you need to use the Portland Group debugger (pgdbg), since our license terms only permit it to be run on the head node.
• To avoid network congestion, under no circumstances should a batch script transfer large amount of data (> 100MB) from the home area. All large data files must be located on /lustre filesystem. Home area is provided for keeping the source files and additional libraries, compiling executables, making plots, etc. All analysis of large data files must be done on /lustre filesystem.
• Special queues (huge, large, and inf) are for special uses. Please do not use the huge queue if you need less than 32GB of memory, do not use large queues if you need less than 16GB of memory, and do not use the inf queue for non-MPI jobs or jobs that use less than 4 nodes. One exception is permitted: you need interactive access to a single node for less than 2 hours, and all other nodes are in use.