Every project proposal is granted a number of compute and storage resources. Once these resources have been fully consumed, no further computations are possible, so it is important to keep an eye on your project's resource consumption.
The steering committe will grant your project a number of CPU hours and GPU hours when the proposal is accepted.
The granted and used amount of CPU/GPU hours can be queried using the following command:
The default output has the following format:
Project Resource Granted Used hkt-project-test cpu 1000000 20000 hkt-project-test gres/gpu 5000 1000
If the amount of resources for a project is unlimited, the
Granted column will show
inf. If a project has been restricted (e.g. it was not accepted by the steering committee while in the preliminary phase), this column will show
0 instead. If the calling account is a member of multiple projects, the output will containe two lines (CPU/GPU resources) for every project.
When the sum of the used CPU or GPU hours of a project is higher or equal than the granted number CPU or GPU hours, new batch jobs will no longer start. Instead they will be held in the
PENDING state and the error message
AssocGrpTRESLimit will be shown. Please see the matching FAQ entry for what to do in this case
The following command line options can be passed to the
||Display help message|
Accounting of resources
The cluster is running in exclusive mode, which means that Slurm always allocates and accounts whole nodes even if the number of requested cores is lower than the ones available. All project proposals should specify their resource consumption with the assumption that a node has 76 CPU cores, since every node hour on HoreKa will account as 76 CPU hours.
Please refer to the File systems chapter to get more information about storage quotas.