| .TH sreport "1" "Slurm Commands" "August 2024" "Slurm Commands" |
| |
| .SH "NAME" |
| sreport \- Generate reports from the slurm accounting data. |
| |
| .SH "SYNOPSIS" |
| \fBsreport\fR [\fIOPTIONS\fR...] [\fICOMMAND\fR...] |
| |
| .SH "DESCRIPTION" |
| \fBsreport\fR is used to generate reports of job usage and cluster |
| utilization for Slurm jobs saved to the Slurm Database, |
| \fBslurmdbd\fR. Report data comes from hourly, daily, and monthly rollups of |
| accounting data that occur automatically in the background. Data will be pulled |
| from the rollup table with the longest interval that can satisfy the requested |
| report period. For example, a report on the time range "Start=01/01 End=03/01" |
| will be able to pull from the monthly rollup table, while a report with |
| "Start=01/01-03:00 End=03/01" will need to use the hourly rollup table. |
| |
| .SH "OPTIONS" |
| |
| .TP |
| \fB\-a\fR, \fB\-\-all_clusters\fR |
| Use all clusters instead of only the cluster from which the command was |
| executed. |
| .IP |
| |
| .TP |
| \fB\-M\fR, \fB\-\-cluster\fR |
| The cluster(s) to generate reports for. Default is local cluster, unless the |
| local cluster is currently part of a federation and in that case generate a |
| report for all clusters in the current federation. If the clusters included |
| in a federation vary through time, use this option to identify the clusters |
| to be included in report. Implies \-\-local. |
| .IP |
| |
| .TP |
| \fB\-\-federation\fR |
| Generate a report for the federation if a member of one. |
| .IP |
| |
| .TP |
| \fB\-h\fR, \fB\-\-help\fR |
| Print a help message describing the usage of \fBsreport\fR. |
| .IP |
| |
| .TP |
| \fB\-\-local\fR |
| Generate a report for the local cluster, even if part of a federation. |
| Overrides \fB\-\-federation\fR. |
| .IP |
| |
| .TP |
| \fB\-n\fR, \fB\-\-noheader\fR |
| Don't display header when listing results. |
| .IP |
| |
| .TP |
| \fB\-p\fR, \fB\-\-parsable\fR |
| Output will be '|' delimited with a '|' at the end. |
| .IP |
| |
| .TP |
| \fB\-P\fR, \fB\-\-parsable2\fR |
| Output will be '|' delimited without a '|' at the end. |
| .IP |
| |
| .TP |
| \fB\-Q\fR, \fB\-\-quiet\fR |
| Print no warning or informational messages, only error messages. |
| .IP |
| |
| .TP |
| \fB\-t\fR <\fItime_format\fR> |
| Specify the output time format. Time format options are case |
| insensitive and may be abbreviated. The default format is Minutes. |
| Supported time format options are listed in the \fBtime\fP command |
| section below. |
| .IP |
| |
| .TP |
| \fB\-T\fR, \fB\-\-tres\fR=<\fItres_names\fR> |
| Trackable resource (TRES) to report values for. |
| By default CPU resource use is reported (except for reservation reports. All |
| TRES types reserved by a reservation will be shown by default unless overridden |
| with this option). |
| Multiple TRES names may be separated using a comma separated list for all |
| reports except the job reports, which can only support a single TRES name, or |
| "ALL" for all TRES. |
| |
| The "Reported" Billing TRES is calculated from the largest Billing TRES of each |
| node multiplied by the time frame. For example, if a node is part of multiple |
| partitions and each has a different TRESBillingWeights defined the Billing TRES |
| for the node will be the highest of the partitions. If TRESBillingWeights is |
| not defined on any partition for a node then the Billing TRES will be equal to |
| the number of CPUs on the node. |
| |
| TRES node usage is no longer reported in percent format or in Cluster |
| Utilization. Please use TRES CPU instead. |
| The main issue with using node is in most configurations multiple jobs are able |
| to run on the same node. This makes TRES node accounting count the same node |
| multiple times in the same period. In exclusive node configurations, CPU |
| accounting returns the same usage node accounting would. |
| .IP |
| |
| .TP |
| \fB\-v\fR, \fB\-\-verbose\fR |
| Print detailed event logging. |
| .IP |
| |
| .TP |
| \fB\-V\fR , \fB\-\-version\fR |
| Print version information and exit. |
| .IP |
| |
| .SH "COMMANDS" |
| |
| .LP |
| \<keyword\> may be omitted from the execute line and sreport will |
| execute in interactive mode. sreport will process commands as entered until |
| explicitly terminated. |
| |
| .TP |
| \fBexit\fP |
| Terminate the execution of sreport. |
| Identical to the \fBquit\fR command. |
| .IP |
| |
| .TP |
| \fBhelp\fP |
| Display a description of sreport options and commands. |
| .IP |
| |
| .TP |
| \fBnonparsable\fP |
| Return output to normal after parsable or parsable2 has been set. |
| .IP |
| |
| .TP |
| \fBparsable\fP |
| Output will be | delimited with an ending '|'. |
| .IP |
| |
| .TP |
| \fBparsable2\fP |
| Output will be | delimited without an ending '|'. |
| .IP |
| |
| .TP |
| \fBquiet\fP |
| Print no warning or informational messages, only fatal error messages. |
| .IP |
| |
| .TP |
| \fBquit\fP |
| Terminate the execution of sreport. |
| Identical to the \fBexit\fR command. |
| .IP |
| |
| .TP |
| \fBtime <time_format>\fP |
| Specify the output time format. Time format options are case |
| insensitive and may be abbreviated. The default format is Minutes. |
| Supported time format options include: |
| .IP |
| .RS |
| .TP 10 |
| \fBSecPer\fR |
| Seconds/Percentage of Total |
| .IP |
| |
| .TP |
| \fBMinPer\fR |
| Minutes/Percentage of Total |
| .IP |
| |
| .TP |
| \fBHourPer\fR |
| Hours/Percentage of Total |
| .IP |
| |
| .TP |
| \fBSeconds\fR |
| Seconds |
| .IP |
| |
| .TP |
| \fBMinutes\fR |
| Minutes |
| .IP |
| |
| .TP |
| \fBHours\fR |
| Hours |
| .IP |
| |
| .TP |
| \fBPercent\fR |
| Percentage of Total |
| .RE |
| .IP |
| |
| .TP |
| \fBverbose\fP |
| Enable detailed event logging. |
| .IP |
| |
| .TP |
| \fBversion\fP |
| Display the sreport version number. |
| .IP |
| |
| .TP |
| \fB!!\fP |
| Repeat the last command executed. |
| .IP |
| |
| .SH "REPORT TYPES" |
| |
| Valid report types are: |
| .RS |
| .TP |
| \fBcluster\fP \<REPORT\> \<OPTIONS\> |
| .IP |
| |
| .TP |
| \fBjob\fP \<REPORT\> \<OPTIONS\> |
| .IP |
| |
| .TP |
| \fBreservation\fP \<REPORT\> \<OPTIONS\> |
| .IP |
| |
| .TP |
| \fBuser\fP \<REPORT\> \<OPTIONS\> |
| .IP |
| .RE |
| |
| \<REPORT\> options for each type include: |
| .RS |
| .TP |
| \fBcluster\fR |
| AccountUtilizationByUser, AccountUtilizationByQOS, UserUtilizationByAccount, |
| UserUtilizationByWckey, Utilization, WCKeyUtilizationByUser |
| .IP |
| |
| .TP |
| \fBjob\fR |
| SizesByAccount, SizesByAccountAndWcKey, SizesByWckey |
| .IP |
| |
| .TP |
| \fBreservation\fR |
| Utilization |
| .IP |
| |
| .TP |
| \fBuser\fR |
| TopUsage |
| .IP |
| .RE |
| |
| .LP |
| \fBNOTE\fR: If \fBOverSubscribe\fR is configured to \fBFORCE\fR or \fBYES\fR |
| in your slurm.conf and the system is not configured to use preemption |
| (\fBPreemptMode=OFF\fR) accounting can easily grow to values greater than |
| the actual utilization. It may be common on such systems to get error messages |
| in the slurmdbd log stating: "We have more allocated time than is possible." |
| |
| .SH "REPORT DESCRIPTION" |
| |
| .TP |
| .B cluster AccountUtilizationByUser |
| This report will display account utilization as it appears on the |
| hierarchical tree. Starting with the specified account or the |
| root account by default this report will list the underlying |
| usage with a sum on each level. Use the 'tree' option to span |
| the tree for better visibility. |
| |
| \fBNOTE\fR: Idle reservation time will be split evenly among accounts/users |
| given access to it. When a reservation is assigned to whole accounts, the |
| time will be counted in the association for the accounts, not the user |
| associations in the accounts. In this case, the usage of a parent account can |
| be larger than the sum of its children. |
| .IP |
| |
| .TP |
| .B cluster AccountUtilizationByQOS |
| This report will display account utilization as it appears on the |
| hierarchical tree. Starting with the root account by default, or with the |
| specified account, this report will list the underlying usage in each |
| account, with a sum on each level. Use the 'tree' option to expand |
| the tree for better visibility. Users are not displayed here, only parent |
| accounts. |
| |
| \fBNOTE\fR: Idle reservation time will be split evenly among accounts that have |
| access to it. Since no QOS is directly connected with the idle time, the |
| time is counted against the default QOS of the account, or 'normal' when |
| there is no default. |
| |
| \fBNOTE\fR: If you reroll data in the database and have changed the default |
| QOS on an association to something other than what was there at the time |
| of the original rollup, you can get different data. |
| |
| .TP |
| .B cluster UserUtilizationByAccount |
| This report will display users by account in order of utilization without |
| grouping multiple accounts by user into one, but displaying them |
| on separate lines. |
| .IP |
| |
| .TP |
| .B cluster UserUtilizationByWCKey |
| This report will display users by wckey in order of utilization without |
| grouping multiple wckey by user into one, but displaying them |
| on separate lines. |
| .IP |
| |
| .TP |
| .B cluster Utilization |
| This report will display total usage divided amongst Allocated, Down, |
| Planned Down, Idle, and Planned time for selected clusters. |
| |
| Refer to the later section for descriptions of these fields. |
| |
| Note: Reservations created with the IGNORE_JOBS flag are not tracked |
| in the Cluster Utilization report due to the fact that allowing any |
| current/active jobs to continue to run in the reservation introduces the |
| possibility for them to be accounted for incorrectly. |
| The jobs in these reservations will be tracked as normal rather than |
| being bundled in the reservation time, as they are with reservations that |
| do not have the IGNORE_JOBS flag. |
| |
| Note: The default view for the "Cluster Utilization" report includes the |
| following fields: Cluster, Allocated, Down, PlannedDown, Idle, Planned, |
| Reported. You can include additional fields like OverCommitted and TresCount |
| fields with the \fBFormat\fP option. The TresName will also be included if |
| using the \fB\-T, \-\-tres <tres_names>\fR option. |
| .RE |
| .IP |
| |
| .TP |
| .B cluster WCKeyUtilizationByUser |
| This report will display wckey utilization sorted by WCKey name for |
| each user on each cluster. |
| .IP |
| |
| .TP |
| .B job SizesByAccount |
| This report will display the amount of time used for job ranges |
| specified by the 'grouping=' option. Only a single level in the tree |
| is displayed defaulting to the root dir. If you specify other |
| accounts with the 'account=' option sreport will use those accounts as |
| the root account and you will receive the aggregated totals of each listed |
| account plus their sub accounts. |
| .IP |
| |
| .TP |
| .B job SizesByAccountAndWckey |
| This report is very similar to SizesByAccount with the difference being |
| each account is pair with wckeys so the identifier is account:wckey |
| instead of just account so there will most likely be multiple accounts |
| listed depending on the number of wckeys used. |
| .IP |
| |
| .TP |
| .B job SizesByWckey |
| This report will display the amount of time for each wckey for job ranges |
| specified by the 'grouping=' option. |
| .IP |
| |
| .TP |
| .B reservation Utilization |
| This report will display total usage for reservations on the systems. |
| Note: Time requests on this report will not truncate the time the reservation |
| used, only the reservations that ran at any time during the period requested. |
| .IP |
| |
| .TP |
| .B user TopUsage |
| Displays the top users on a cluster, i.e. users with the highest usage. |
| By default users are sorted by CPUTime, but the \-T, \-\-tres option will |
| sort users by the first TRES specified. |
| |
| Use the group option to group accounts together. |
| The default is to have a different line for each user account combination. |
| .IP |
| |
| .LP |
| Each report type has various options... |
| |
| .LP |
| .B OPTIONS FOR ALL REPORT TYPES |
| |
| .RS |
| .TP |
| .B All_Clusters |
| Use all monitored clusters. Default is local cluster. |
| .IP |
| |
| .TP |
| .B Clusters=<OPT> |
| List of clusters to include in report. Default is local cluster. |
| .IP |
| |
| .TP |
| .B End=<OPT> |
| Period ending for report. Default is 23:59:59 of previous day. Note that while |
| minutes and seconds are recognized, they will be rounded to the previous hour as |
| reports cannot be generated for a more specific time range. |
| Valid time formats are... |
| .sp |
| HH:MM[:SS] [AM|PM] |
| .br |
| MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] |
| .br |
| MM/DD[/YY]\-HH:MM[:SS] |
| .br |
| YYYY\-MM\-DD[THH:MM[:SS]] |
| .br |
| now[{+|\-}\fIcount\fR[seconds(default)|minutes|hours|days|weeks]] |
| .IP |
| |
| .TP |
| .B Format=<OPT> |
| Comma separated list of fields to display in report. |
| |
| When using the format option for listing various fields you can put a |
| %NUMBER afterwards to specify how many characters should be printed. |
| |
| e.g. format=name%30 will print 30 characters of field name right |
| justified. A \-30 will print 30 characters left justified. |
| .IP |
| |
| .TP |
| .B Start=<OPT> |
| Period start for report. Default is 00:00:00 of previous day. Note that while |
| minutes and seconds are recognized, they will be rounded to the next hour as |
| reports cannot be generated for a more specific time range. |
| Valid time formats are... |
| .sp |
| HH:MM[:SS] [AM|PM] |
| .br |
| MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] |
| .br |
| MM/DD[/YY]\-HH:MM[:SS] |
| .br |
| YYYY\-MM\-DD[THH:MM[:SS]] |
| .br |
| now[{+|\-}\fIcount\fR[seconds(default)|minutes|hours|days|weeks]] |
| .ad |
| .RE |
| .IP |
| |
| .LP |
| .B OPTIONS SPECIFICALLY FOR CLUSTER REPORTS |
| |
| .RS |
| .TP |
| .B Accounts=<OPT> |
| When used with the UserUtilizationByAccount, or |
| AccountUtilizationBy[User|QOS], List of accounts to include in report. |
| Default is all. |
| .IP |
| |
| .TP |
| .B Tree |
| When used with the AccountUtilizationBy[User|QOS] report will span the |
| accounts as they are in the hierarchy. |
| .IP |
| |
| .TP |
| .B Users=<OPT> |
| When used with any report other than Utilization, List of users to |
| include in report. Default is all. |
| .IP |
| |
| .TP |
| .B Wckeys=<OPT> |
| When used with the UserUtilizationByWckey or WCKeyUtilizationByUser, |
| List of wckeys to include in report. Default is all. |
| .RE |
| .IP |
| |
| .LP |
| .B OPTIONS SPECIFICALLY FOR JOB REPORTS |
| |
| .RS |
| .TP |
| .B Accounts=<OPT> |
| List of accounts to use for the report. Default is all which will show only |
| one line corresponding to the totals of all accounts in the hierarchy. |
| This explanation does not apply when ran with the FlatView or AcctAsParent |
| options. |
| .IP |
| |
| .TP |
| .B AcctAsParent |
| When used with the SizesbyAccount(*) will take specified accounts |
| as parents and the next layer of accounts under those specified |
| will be displayed. Default is root if no specific Accounts are requested. |
| When FlatView is used, this option is ignored. |
| .IP |
| |
| .TP |
| .B FlatView |
| When used with the SizesbyAccount(*) will not group accounts in a |
| hierarchical level, but print each account where jobs ran on a |
| separate line without any hierarchy. |
| .IP |
| |
| .TP |
| .B GID=<OPT> |
| List of group ids to include in report. Default is all. |
| .IP |
| |
| .TP |
| .B Grouping=<OPT> |
| Comma separated list of size groupings. (e.g. 50,100,150 would group |
| job cpu count 1\-49, 50\-99, 100\-149, > 150). grouping=individual will |
| result in a single column for each job size found. |
| .IP |
| |
| .TP |
| .B Jobs=<OPT> |
| List of jobs/steps to include in report. Default is all. |
| .IP |
| |
| .TP |
| .B Nodes=<OPT> |
| Only show jobs that ran on these nodes. Default is all. |
| .IP |
| |
| .TP |
| .B Partitions=<OPT> |
| List of partitions jobs ran on to include in report. Default is all. |
| .IP |
| |
| .TP |
| .B PrintJobCount |
| When used with the Sizes report will print number of jobs ran instead |
| of time used. |
| .IP |
| |
| .TP |
| .B Users=<OPT> |
| List of users jobs to include in report. Default is all. |
| .IP |
| |
| .TP |
| .B Wckeys=<OPT> |
| List of wckeys to use for the report. Default is all. The |
| SizesbyWckey report all users summed together. If you want only |
| certain users specify them with the Users= option. |
| .RE |
| .IP |
| |
| .LP |
| .B OPTIONS SPECIFICALLY FOR RESERVATION REPORTS |
| |
| .RS |
| .TP |
| .B Names=<OPT> |
| List of reservations to use for the report. Default is all. |
| .IP |
| |
| .TP |
| .B Nodes=<OPT> |
| Only show reservations that used these nodes. Default is all. |
| .RE |
| .IP |
| |
| .LP |
| .B OPTIONS SPECIFICALLY FOR USER REPORTS |
| |
| .RS |
| .TP |
| .B Accounts=<OPT> |
| List of accounts to use for the report. Default is all. |
| .IP |
| |
| .TP |
| .B Group |
| Group all accounts together for each user. Default is a separate |
| entry for each user and account reference. |
| .IP |
| |
| .TP |
| .B TopCount=<OPT> |
| Used in the TopUsage report. Change the number of users displayed. |
| Default is 10. |
| .IP |
| |
| .TP |
| .B Users=<OPT> |
| List of users jobs to include in report. Default is all. |
| .RE |
| .IP |
| |
| .SH "FORMAT OPTIONS FOR EACH REPORT" |
| |
| .LP |
| \fBFORMAT OPTIONS FOR CLUSTER REPORTS\fP |
| .RS |
| .TP |
| \fBAccountUtilizationByUser\fR |
| Accounts, Cluster, Login, QOS, Proper, TresCount, Used |
| .IP |
| |
| .TP |
| \fBAccountUtilizationByQOS\fR |
| Accounts, Cluster, QOS, TresCount, Used |
| .IP |
| |
| .TP |
| \fBUserUtilizationByAccount\fR |
| Accounts, Cluster, Login, Proper, TresCount, Used |
| .IP |
| |
| .TP |
| \fBUserUtilizationByWckey\fR |
| Cluster, Login, Proper, TresCount, Used, Wckey |
| .IP |
| |
| .TP |
| \fBUtilization\fR |
| Allocated, Cluster, Down, Idle, OverCommitted, PlannedDown, Reported, Planned, |
| TresCount, TresName |
| .IP |
| |
| .TP |
| \fBWCKeyUtilizationByUser\fR |
| Cluster, Login, Proper, TresCount, Used, Wckey |
| .IP |
| .RE |
| |
| .LP |
| \fBFORMAT OPTIONS FOR JOB REPORTS\fP |
| |
| .RS |
| .TP |
| \fBSizesByAccount\fR |
| Account, Cluster |
| .IP |
| |
| .TP |
| \fBSizesByAccountAndWckey\fR |
| Account, Cluster |
| .IP |
| |
| .TP |
| \fBSizesByWckey\fR |
| Wckey, Cluster |
| .IP |
| .RE |
| |
| .LP |
| \fBFORMAT OPTIONS FOR RESERVATION REPORTS\fP |
| |
| .RS |
| .TP |
| \fBUtilization\fR |
| Allocated, Associations, Cluster, End, Flags, Idle, Name, Nodes, ReservationId, Start, TotalTime, TresCount, TresName, TresTime |
| .IP |
| .RE |
| |
| .LP |
| \fBFORMAT OPTIONS FOR USER REPORTS\fP |
| |
| .RS |
| .TP |
| \fBTopUsage\fR |
| Account, Cluster, Login, Proper, Used |
| .IP |
| .RE |
| |
| .LP |
| All commands and options are case\-insensitive. |
| |
| .SH "EXPLANATION OF REPORT FIELDS" |
| .LP |
| .TP |
| \fBAccount\fP |
| Account name |
| .IP |
| .TP |
| \fBAllocated\fP |
| \fBCluster utilization report:\fR Time that nodes were in use with active jobs or an |
| active reservation. This does not include reservations created with the MAINT or |
| IGNORE_JOBS flags. |
| |
| \fBReservation reports:\fR Time that nodes were in use with active jobs in this |
| reservation |
| .IP |
| .TP |
| \fBAssociations\fP |
| Associations allowed in the reservation |
| .IP |
| .TP |
| \fBCluster\fP |
| Cluster name |
| .IP |
| .TP |
| \fBDown\fP |
| \fBCluster utilization report only:\fR Time that nodes were marked as Down or fully |
| Drained, or time that slurmctld was not responding (if TrackSlurmctldDown is set |
| in slurmdbd.conf) |
| .IP |
| .TP |
| \fBEnd\fP |
| \fBReservation reports only:\fR End time |
| .IP |
| .TP |
| \fBEnergy\fP |
| Energy use (if tracking energy consumption) |
| .IP |
| .TP |
| \fBFlags\fP |
| \fBReservation reports only:\fR Flags applied to the reservation |
| .IP |
| .TP |
| \fBIdle\fP |
| \fBCluster utilization report:\fR Time that nodes were not Allocated, Down, |
| PlannedDown, or Planned |
| |
| \fBReservation reports:\fR Time that nodes were not Allocated |
| .IP |
| .TP |
| \fBLogin\fP |
| User's login name |
| .IP |
| .TP |
| \fBName\fP |
| \fBReservation reports only:\fR Reservation name |
| .IP |
| .TP |
| \fBOverCommitted\fP |
| \fBCluster utilization report only:\fR Time of eligible jobs waiting in the |
| queue over the Planned time. This contains any overflow of Planned time that |
| would otherwise exceed the Reported time. It is typically useful to |
| determine whether your system is overloaded and by how much. |
| .IP |
| .TP |
| \fBPlanned\fP |
| \fBCluster utilization report only:\fR Time that nodes were not Allocated, Down or |
| PlannedDown with eligible jobs in the queue that were unable to start due to |
| time or size constraints. If Allocated plus Planned time exceeds the Reported |
| time, the excess will be reported as OverCommitted. If this value is not of |
| importance for you then the number can be grouped with idle time. |
| .IP |
| .TP |
| \fBPlannedDown\fP |
| \fBCluster utilization report only:\fR Time that nodes were in use by a reservation |
| created with the MAINT flag but not the IGNORE_JOBS flag. Also, time that nodes |
| were in the FUTURE or POWERED_DOWN state. |
| .IP |
| .TP |
| \fBProper Name\fP |
| User's proper/real name |
| .IP |
| .TP |
| \fBReported\fP |
| \fBCluster utilization report only:\fR Total time that records are available for. |
| Nodes that were added to or removed from the cluster during the report period |
| will only contribute usage data for the time they were present in the cluster |
| and cause percentage calculations to deviate from 100%. |
| .IP |
| .TP |
| \fBReservationId\fP |
| \fBReservation reports only:\fR Reservation ID |
| .IP |
| .TP |
| \fBStart\fP |
| \fBReservation reports only:\fR Start time |
| .IP |
| .TP |
| \fBTotalTime\fP |
| \fBReservation reports only:\fR Amount of time the reservation was valid |
| .IP |
| .TP |
| \fBTresCount\fP |
| Number of TRES present |
| .IP |
| .TP |
| \fBTresName\fP |
| Name of TRES reported |
| .IP |
| .TP |
| \fBTresTime\fP |
| \fBReservation reports only:\fR Total TRES-time in the reservation |
| .IP |
| .TP |
| \fBUsed\fP |
| Used TRES-minutes |
| .IP |
| .TP |
| \fBWckey\fP |
| Workload Characterization Key |
| .IP |
| |
| .SH "PERFORMANCE" |
| .PP |
| Executing \fBsreport\fR sends a remote procedure call to \fBslurmdbd\fR. If |
| enough calls from \fBsreport\fR or other Slurm client commands that send remote |
| procedure calls to the \fBslurmdbd\fR daemon come in at once, it can result in a |
| degradation of performance of the \fBslurmdbd\fR daemon, possibly resulting in a |
| denial of service. |
| .PP |
| Do not run \fBsreport\fR or other Slurm client commands that send remote |
| procedure calls to \fBslurmdbd\fR from loops in shell scripts or other programs. |
| Ensure that programs limit calls to \fBsreport\fR to the minimum necessary for |
| the information you are trying to gather. |
| |
| .SH "ENVIRONMENT VARIABLES" |
| .PP |
| Some \fBsreport\fR options may be set via environment variables. These |
| environment variables, along with their corresponding options, are listed below. |
| (Note: Command line options will always override these settings.) |
| .IP |
| |
| .TP 20 |
| \fBSREPORT_CLUSTER\fR |
| Same as \fB\-M\fR, \fB\-\-cluster\fR |
| .IP |
| |
| .TP |
| \fBSREPORT_FEDERATION\fR |
| Same as \-\-federation\fR |
| .IP |
| |
| .TP |
| \fBSREPORT_LOCAL\fR |
| Same as \-\-local\fR |
| .IP |
| |
| .TP |
| \fBSREPORT_TRES\fR |
| Same as \fB\-t, \-\-tres\fR |
| .IP |
| |
| .TP |
| \fBSLURM_CONF\fR |
| The location of the Slurm configuration file. |
| |
| .in 0 |
| .SH "EXAMPLES" |
| |
| .TP |
| Report number of jobs per account according to different job size bins: |
| .IP |
| .nf |
| $ sreport job sizesbyaccount Start=11:00 |
| -------------------------------------------------------------------------------- |
| Job Sizes 2024-08-19T11:00:00 - 2024-08-19T11:59:59 (3600 secs) |
| Time reported in Minutes |
| -------------------------------------------------------------------------------- |
| Cluster Account 0-49 CPUs 50-249 CPUs 250-499 CPUs 500-999 CPUs >= 1000 CPUs % of cluster |
| --------- --------- ------------- ------------- ------------- ------------- ------------- ------------ |
| minesofm+ root 770 0 0 0 0 100.00% |
| .fi |
| |
| .TP |
| Report cluster utilization: |
| .IP |
| .nf |
| $ sreport cluster utilization Start=11:00 |
| -------------------------------------------------------------------------------- |
| Cluster Utilization 2024-08-19T11:00:00 - 2024-08-19T11:59:59 |
| Usage reported in CPU Minutes |
| -------------------------------------------------------------------------------- |
| Cluster Allocate Down Planned Idle Planned Reported |
| --------- -------- -------- -------- -------- -------- -------- |
| minesofm+ 770 120 239 3791 239 4920 |
| .fi |
| |
| .TP |
| Report top usage: |
| .IP |
| .nf |
| $ sreport user top Start=11:00 |
| -------------------------------------------------------------------------------- |
| Top 10 Users 2024-08-19T11:00:00 - 2024-08-19T11:59:59 (3600 secs) |
| Usage reported in CPU Minutes |
| -------------------------------------------------------------------------------- |
| Cluster Login Proper Name Account Used Energy |
| --------- --------- --------------- --------------- -------- -------- |
| minesofm+ stephen ,,, main 538 0 |
| minesofm+ phteven ,,, main 145 0 |
| minesofm+ phteven ,,, rivendell 45 0 |
| minesofm+ stephen ,,, rivendell 41 0 |
| .fi |
| |
| .TP |
| Report jobs by size from specific user and account: |
| .IP |
| .nf |
| $ sreport job sizesbyaccount All_Clusters users=stephen account=main PrintJobCount Start=11:00 |
| -------------------------------------------------------------------------------- |
| Job Sizes 2024-08-19T11:00:00 - 2024-08-19T11:59:59 (3600 secs) |
| Units are in number of jobs ran |
| -------------------------------------------------------------------------------- |
| Cluster Account 0-49 CPUs 50-249 CPUs 250-499 CPUs 500-999 CPUs >= 1000 CPUs % of cluster |
| --------- --------- ------------- ------------- ------------- ------------- ------------- ------------ |
| minesofm+ main 12 0 0 0 0 100.00% |
| .fi |
| |
| .TP |
| Report cluster account utilization with the specified fields during the |
| specified day and by the specified user: |
| .IP |
| .nf |
| $ sreport cluster AccountUtilizationByUser start=7/10 end=7/11 \ |
| cluster=minesofmoria user=stephen format=Account,Cluster,TresCount,Login,Proper,Used |
| -------------------------------------------------------------------------------- |
| Cluster/Account/User Utilization 2024-07-10T00:00:00 - 2024-07-10T23:59:59 (86400 secs) |
| Usage reported in CPU Minutes |
| -------------------------------------------------------------------------------- |
| Account Cluster TRES Count Login Proper Name Used |
| --------------- --------- ---------- --------- --------------- -------- |
| main minesofm+ stephen ,,, 38 |
| rivendell minesofm+ stephen ,,, 1 |
| arwen minesofm+ stephen ,,, 1 |
| elrond minesofm+ stephen ,,, 1 |
| .fi |
| |
| .TP |
| Report cluster account utilization by user for a specific account during a |
| 1\-week period: |
| .IP |
| .nf |
| $ sreport cluster AccountUtilizationByUser start=7/23 end=7/24 \ |
| cluster=minesofmoria account=main |
| -------------------------------------------------------------------------------- |
| Cluster/Account/User Utilization 2024-07-23T00:00:00 - 2024-07-23T23:59:59 (86400 secs) |
| Usage reported in CPU Minutes |
| -------------------------------------------------------------------------------- |
| Cluster Account Login Proper Name Used Energy |
| --------- --------------- --------- --------------- -------- -------- |
| minesofm+ main 148 0 |
| minesofm+ main phteven ,,, 2 0 |
| minesofm+ main stephen ,,, 146 0 |
| .fi |
| |
| .TP |
| Report top usage in percent for a specific account: |
| .IP |
| .nf |
| $ sreport user topusage start=11:00 -t percent account=main |
| -------------------------------------------------------------------------------- |
| Top 10 Users 2024-08-19T11:00:00 - 2024-08-19T11:59:59 (3600 secs) |
| Usage reported in Percentage of Total |
| -------------------------------------------------------------------------------- |
| Cluster Login Proper Name Account Used Energy |
| --------- --------- --------------- --------------- -------- -------- |
| minesofm+ stephen ,,, main 10.94% 0.00% |
| minesofm+ phteven ,,, main 2.95% 0.00% |
| .fi |
| |
| .SH "COPYING" |
| Copyright (C) 2009\-2010 Lawrence Livermore National Security. |
| Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). |
| .br |
| Copyright (C) 2010\-2022 SchedMD LLC. |
| .LP |
| This file is part of Slurm, a resource management program. |
| For details, see <https://slurm.schedmd.com/>. |
| .LP |
| Slurm is free software; you can redistribute it and/or modify it under |
| the terms of the GNU General Public License as published by the Free |
| Software Foundation; either version 2 of the License, or (at your option) |
| any later version. |
| .LP |
| Slurm is distributed in the hope that it will be useful, but WITHOUT ANY |
| WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS |
| FOR A PARTICULAR PURPOSE. See the GNU General Public License for more |
| details. |
| |
| .SH "SEE ALSO" |
| \fBsacct\fR(1), \fBslurmdbd\fR(8) |