<!--#include virtual="header.txt"-->

<h1> CPU Management User and Administrator Guide</h1>
<a name="Overview"></a>
<h2>Overview</h2>
<p>The purpose of this guide is to assist SLURM users and administrators in selecting configuration options 
and composing command lines to manage the use of CPU resources by jobs, steps and tasks. The document 
is divided into the following sections:</p>
<ul>
<li><a href="#Overview">Overview</a></li>
<li><a href="#Section1">CPU Management Steps performed by SLURM</a></li>
<li><a href="#Section2">Getting Information about CPU usage by Jobs/Steps/Tasks</a></li>
<li><a href="#Section3">CPU Management and SLURM Accounting</a></li>
<li><a href="#Section4">CPU Management Examples</a></li>
</ul>

<p>CPU Management through user commands is constrained by the configuration parameters
chosen by the SLURM administrator. The interactions between different CPU management options are complex 
and often difficult to predict. Some experimentation may be required to discover the exact combination 
of options  needed to produce a desired outcome. Users and administrators should refer to the man pages 
for <a href="slurm.conf.html">slurm.conf</a>, <a href="cgroup.conf.html">cgroup.conf</a>,
<a href="salloc.html">salloc</a>, 
<a href="sbatch.html">sbatch</a> and <a href="srun.html">srun</a> for detailed explanations of each 
option. The following html documents may also be useful:</p>

<p>
<a href="cons_res.html">Consumable Resources in SLURM</a><br>
<a href="cons_res_share.html">Sharing Consumable Resources</a><br>
<a href="mc_support.html">Support for Multi-core/Multi-thread 
Architectures</a><br>
<a href="dist_plane.html">Plane distribution</a></p>

<p>This document describes SLURM CPU management for conventional Linux clusters only.  For
information on Cray and IBM BlueGene systems, please refer to the appropriate documents.</p>
<p>The information and examples in this document have been verified on SLURM version 2.4.0. Some
information may not be valid for previous SLURM versions.</p><br>
<a name="Section1"></a>
<h2>CPU Management Steps performed by SLURM</h2>
<p>SLURM uses four basic steps to manage CPU resources for a job/step:</p>
<ul>
<li><a href="#Step1">Step 1: Selection of Nodes</a>
</li><li><a href="#Step2">Step 2: Allocation of CPUs from the selected Nodes</a>
</li><li><a href="#Step3">Step 3: Distribution of Tasks to the selected Nodes</a>
</li><li><a href="#Step4">Step 4: Optional Distribution and Binding of Tasks to CPUs within a Node 
</a>
</li></ul>
<a name="Step1"></a>
<h3>Step 1: Selection of Nodes</h3>
<p>In Step 1, SLURM selects the set of nodes from which CPU resources are to be allocated to a job or 
job step.  Node selection is therefore influenced by many of the configuration and command line options
that control the allocation of CPUs (Step 2 below).
If <font face="Courier New, monospace">
SelectType=select/linear</font> is configured, all resources on the selected nodes will be allocated
to the job/step. If <font face="Courier New, monospace">SelectType=select/cons_res</font> is configured, 
individual sockets, cores and threads may be allocated from the selected nodes as 
<a href="cons_res.html">consumable resources</a>. The consumable resource type is defined by 
<font face="Courier New, monospace">SelectTypeParameters.</font>
<br>
<br>
Step 1 is performed by slurmctld and the select plugin.
<br>
</p><center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>slurm.conf options that control Step 1</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>slurm.conf
				parameter</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Possible values</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_NodeName">NodeName</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;name of the node&gt;<br><br>
				Plus additional parameters. See man page for details.</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Defines
				a node. This includes the number and layout of boards, sockets, cores,
				threads and processors (logical CPUs) on the node.</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_PartitionName">PartitionName</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;name of the partition&gt;<br><br>
				Plus additional parameters. See man page for details.</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Defines
				a partition. Several parameters of the partition definition
				affect the selection of nodes (e.g., Nodes,
				Shared, MaxNodes)</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_FastSchedule">FastSchedule</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">0 | 1 | 2</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Controls
				how the information in a node definition is used.</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_SelectType">SelectType</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
                               <font face="Courier New, monospace">select/linear | select/cons_res</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Controls
				whether CPU resources are allocated to jobs and job steps in
				units of whole nodes or as consumable resources (sockets, cores
				or threads).</font></p>
			</td>
		</tr>
		<tr>
			<td height="17" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_SelectTypeParameters">SelectTypeParameters</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">CR_CPU | CR_CPU_Memory | CR_Core |
CR_Core_Memory | CR_Socket | CR_Socket_Memory</font><br><br>Plus additional options.  See man page for details.</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Defines
				the consumable resource type and controls other aspects of CPU
				resource allocation by the select plugin.</font></p>
			</td>
		</tr>
	</tbody></table>
</center> 
<br>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>srun/salloc/sbatch command line options that control Step 1</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="17" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Command
				line option</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Possible values</b></font></p>
			</td><td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_extra-node-info">-B, --extra-node-info</a>
				</font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;sockets[:cores[:threads]]&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with a specified layout of sockets, cores
				and threads.</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_constraint">-C, --constraint</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;list&gt;
				</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with specified attributes</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_contiguous">--contiguous</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to contiguous nodes</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_cores-per-socket">--cores-per-socket</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;cores&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with at least the specified number of cores per socket</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_cpus-per-task">-c, --cpus-per-task</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;ncpus&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the number of CPUs allocated per task</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_exclusive">--exclusive</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Prevents
				sharing of allocated nodes with other jobs. Suballocates CPUs to job steps.</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="salloc.html#OPT_nodefile">-F, --nodefile</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;node file&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">File
				containing a list of specific nodes to be selected for the job (salloc and sbatch only)</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_hint">--hint</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">compute_bound |
                                memory_bound | [no]multithread</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Additional
				controls on allocation of CPU resources</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_mincpus">--mincpus</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;n&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the minimum number of CPUs allocated per node</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_nodes">-N, --nodes</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;minnodes[-maxnodes]&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the minimum/maximum number of nodes allocated to the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks">-n, --ntasks</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the number of tasks to be created for the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-core">--ntasks-per-core</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the maximum number of tasks per allocated core</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-socket">--ntasks-per-socket</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the maximum number of tasks per allocated socket</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-node">--ntasks-per-node</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the maximum number of tasks per allocated node</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_overcommit">-O, --overcommit</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Allows
				fewer CPUs to be allocated than the number of tasks</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_partition">-p, --partition</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;partition_names&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				which partition is used for the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_share">-s, --share</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Allows
				sharing of allocated nodes with other jobs</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_sockets-per-node">--sockets-per-node</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;sockets&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with at least the specified number of sockets</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_threads-per-core">--threads-per-core</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;threads&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with at least the specified number of threads per core</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_nodelist">-w, --nodelist</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;host1,host2,... or filename&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">List
				of specific nodes to be allocated to the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_exclude">-x, --exclude</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
                                &lt;host1,host2,... or filename&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">List
				of specific nodes to be excluded from allocation to the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="17" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_no-allocate">-Z, --no-allocate</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Bypass
				normal allocation (privileged option available to users
				&#8220;SlurmUser&#8221; and &#8220;root&#8221; only)</font></font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Step2"></a>
<h3>Step 2: Allocation of CPUs from the selected Nodes</h3>
<p>In Step 2, SLURM allocates CPU resources to a job/step from the set of nodes selected 
in Step 1. CPU allocation is therefore influenced by the configuration and command line options
that relate to node selection.
If <font face="Courier New, monospace">
SelectType=select/linear</font> is configured, all resources on the selected nodes will be allocated
to the job/step. If <font face="Courier New, monospace">SelectType=select/cons_res</font> is configured, 
individual sockets, cores and threads may be allocated from the selected nodes as 
<a href="cons_res.html">consumable resources</a>. The consumable resource type is defined by 
<font face="Courier New, monospace">SelectTypeParameters.</font>
<br>
</p><p>When using <font face="Courier New, monospace">SelectType=select/cons_res</font>, 
the default allocation method across nodes is block allocation (allocate all available CPUs in 
a node before using another node). The default allocation method within a node is cyclic 
allocation (allocate available CPUs in a round-robin fashion across the sockets within a node). 
Users may override the default behavior using the appropriate command 
line options described below.  The choice of allocation methods may influence which specific
CPUs are allocated to the job/step.
<br><br>
Step 2 is performed by slurmctld and the select plugin.
<br>
</p><center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>slurm.conf options that control Step 2</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>slurm.conf
				parameter</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Possible values</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_NodeName">NodeName</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;name of the node&gt;<br><br>
				Plus additional parameters. See man page for details.</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Defines
				a node. This includes the number and layout of boards, sockets, cores,
				threads and processors (logical CPUs) on the node.</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_PartitionName">PartitionName</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;name of the partition&gt;<br><br>
				Plus additional parameters. See man page for details.</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Defines
				a partition. Several parameters of the partition definition
				affect the allocation of CPU resources to jobs (e.g., Nodes,
				Shared, MaxNodes)</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_FastSchedule">FastSchedule</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">0 | 1 | 2</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Controls
				how the information in a node definition is used.</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_SelectType">SelectType</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
                               <font face="Courier New, monospace">select/linear | select/cons_res</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Controls
				whether CPU resources are allocated to jobs and job steps in
				units of whole nodes or as consumable resources (sockets, cores
				or threads).</font></p>
			</td>
		</tr>
		<tr>
			<td height="17" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_SelectTypeParameters">SelectTypeParameters</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">CR_CPU | CR_CPU_Memory | CR_Core |
CR_Core_Memory | CR_Socket | CR_Socket_Memory</font><br><br>Plus additional options.  See man page for details.</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Defines
				the consumable resource type and controls other aspects of CPU
				resource allocation by the select plugin.</font></p>
			</td>
		</tr>
	</tbody></table>
</center> 
<br>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>srun/salloc/sbatch command line options that control Step 2</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="17" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Command
				line option</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Possible values</b></font></p>
			</td><td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_extra-node-info">-B, --extra-node-info</a>
				</font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;sockets[:cores[:threads]]&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with a specified layout of sockets, cores
				and threads.</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_constraint">-C, --constraint</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;list&gt;
				</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with specified attributes</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_contiguous">--contiguous</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to contiguous nodes</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_cores-per-socket">--cores-per-socket</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;cores&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with at least the specified number of cores per socket</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_cpus-per-task">-c, --cpus-per-task</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;ncpus&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the number of CPUs allocated per task</font></font></p>
			</td>
		</tr>
				<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_distribution">--distribution, -m</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">
				block|cyclic<br>|arbitrary|plane=&lt;options&gt;[:block|cyclic]</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">The second specified distribution (after the ":")
				can be used to override the default allocation method within nodes</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_exclusive">--exclusive</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Prevents
				sharing of allocated nodes with other jobs</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="salloc.html#OPT_nodefile">-F, --nodefile</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;node file&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">File
				containing a list of specific nodes to be selected for the job (salloc and sbatch only)</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_hint">--hint</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">compute_bound |
                                memory_bound | [no]multithread</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Additional
				controls on allocation of CPU resources</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_mincpus">--mincpus</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;n&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the minimum number of CPUs allocated per node</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_nodes">-N, --nodes</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;minnodes[-maxnodes]&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the minimum/maximum number of nodes allocated to the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks">-n, --ntasks</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the number of tasks to be created for the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-core">--ntasks-per-core</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the maximum number of tasks per allocated core</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-socket">--ntasks-per-socket</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the maximum number of tasks per allocated socket</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-node">--ntasks-per-node</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				the maximum number of tasks per allocated node</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_overcommit">-O, --overcommit</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Allows
				fewer CPUs to be allocated than the number of tasks</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_partition">-p, --partition</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;partition_names&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				which partition is used for the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_share">-s, --share</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Allows
				sharing of allocated nodes with other jobs</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_sockets-per-node">--sockets-per-node</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;sockets&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with at least the specified number of sockets</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_threads-per-core">--threads-per-core</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;threads&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Restricts
				node selection to nodes with at least the specified number of threads per core</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_nodelist">-w, --nodelist</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				&lt;host1,host2,... or filename&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">List
				of specific nodes to be allocated to the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_exclude">-x, --exclude</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
                                &lt;host1,host2,... or filename&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">List
				of specific nodes to be excluded from allocation to the job</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="17" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_no-allocate">-Z, --no-allocate</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Bypass
				normal allocation (privileged option available to users
				&#8220;SlurmUser&#8221; and &#8220;root&#8221; only)</font></font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Step3"></a>
<h3>Step 3: Distribution of Tasks to the selected Nodes</h3>
<p>In Step 3, SLURM distributes tasks to the nodes that were selected for 
the job/step in Step 1. Each task is distributed to only one node, but more than one 
task may be distributed to each node.  Unless overcommitment of CPUs to tasks is
specified for the job, the number of tasks distributed to a node is
constrained by the number of CPUs allocated on the node and the number of CPUs per
task. If consumable resources is configured, or resource sharing is allowed, tasks from 
more than one job/step may run on the same node concurrently.  
<br><br>
Step 3 is performed by slurmctld.
<br>
</p><center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>slurm.conf options that control Step 3</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>slurm.conf
				parameter</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Possible values</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_MaxTasksPerNode">MaxTasksPerNode</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				Controls the maximum number of tasks that a job step can spawn on a single node
				</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>srun/salloc/sbatch command line options that control Step 3</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="17" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Command
				line option</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Possible values</b></font></p>
			</td><td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_distribution">--distribution, -m</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">
				block|cyclic<br>|arbitrary|plane=&lt;options&gt;[:block|cyclic]</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">The first specified distribution (before the ":")
				controls the sequence in which tasks are distributed to each of the selected nodes. Note that 
				this option does not affect the number of tasks distributed to each node, but only the sequence of 
				distribution.</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-core">--ntasks-per-core</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;
				</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">
				Controls the maximum number of tasks per allocated core</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-socket">--ntasks-per-socket</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">
				Controls the maximum number of tasks per allocated socket</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-node">--ntasks-per-node</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">
				Controls the maximum number of tasks per allocated node</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_relative">-r, --relative</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">N/A</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">Controls
				which node is used for a job step</font></font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Step4"></a>
<h3>Step 4: Optional Distribution and Binding of Tasks to CPUs within a Node</h3>
<p>In optional Step 4, SLURM distributes and binds each task to a specified subset of 
the allocated CPUs on the node to which the task was distributed in Step 3. Different 
tasks distributed to the same node may be bound to the same subset of CPUs or to 
different subsets. This step is known as task affinity or task/CPU binding.
<br><br>
Step 4 is performed by slurmd and the task plugin.
<br>
</p><center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>slurm.conf options that control Step 4</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>slurm.conf
				parameter</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Possible values</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_TaskPlugin">TaskPlugin</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">
				task/none | task/affinity | task/cgroup</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				Controls whether this step is enabled and which task plugin to use
				</font></p>
			</td>
		</tr>
				<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="slurm.conf.html#OPT_TaskPluginParam">TaskPluginParam</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">See man page</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				For task/affinity, controls the binding unit (sockets, cores or threads) and the 
                                binding method (sched or cpusets)</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
</p><center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>cgroup.conf options that control Step 4 (task/cgroup plugin only)</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>cgroup.conf
				parameter</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" height="18" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Possible values</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="cgroup.conf.html">ConstrainCores</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">
				yes|no</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				Controls whether jobs are constrained to their allocated CPUs
				</font></p>
			</td>
		</tr>
				<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="cgroup.conf.html">TaskAffinity</a></font></font></p>
			</td>
			<td height="18" width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">
				yes|no</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				Controls whether task-to-CPU binding is enabled</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="100%">
                <caption style="font-size: 8pt"><b>srun/salloc/sbatch command line options that control Step 4</b></caption>
		<colgroup><col width="20%">
		<col width="20%">
		<col width="60%">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="17" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Command
				line option</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Possible values</b></font></p>
			</td><td bgcolor="#e0e0e0" width="60%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
                    <b style="font-family: Arial, Helvetica, sans-serif">Description</b></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_cpu_bind">--cpu_bind</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
				See man page</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">Controls binding of tasks to CPUs</font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_ntasks-per-core">--ntasks-per-core</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">&lt;number&gt;
				</font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">
				Controls the maximum number of tasks per allocated core</font></font></p>
			</td>
		</tr>
		<tr>
			<td height="18" width="20%">
				<p align="LEFT"><font face="Courier New, monospace"><font style="font-size: 8pt" size="1">
				<a href="srun.html#OPT_distribution">--distribution, -m</a></font></font></p>
			</td>
			<td width="20%">
				<p align="LEFT"><font style="font-size: 8pt" size="1"><font face="Courier New, monospace">
				block|cyclic<br>|arbitrary|plane=&lt;options&gt;[:block|cyclic]</font></font></p>
			</td>
			<td width="60%">
				<p align="LEFT"><font face="Arial, sans-serif"><font style="font-size: 8pt" size="1">
				The second specified distribution (after the ":") controls the sequence in which tasks are 
				distributed to allocated CPUs within a node for binding of tasks to CPUs</font></font></p>
			</td>
		</tr>

	</tbody></table>
</center>
<br><br>
<h2>Additional Notes on CPU Management Steps</h2>
<p>For consumable resources, it is important for users to understand the difference between 
cpu allocation (Step 2) and task affinity/binding (Step 4).  Exclusive (unshared) allocation 
of CPUs as consumable resources limits the number of jobs/steps/tasks that 
can use a node concurrently.  But it does not limit the set of CPUs on the node that each 
task distributed to the node can use.  Unless some form of CPU/task binding is used 
(e.g., a task or spank plugin), all tasks distributed to a node can use all of 
the CPUs on the node, including CPUs not allocated to their job/step.  This may have 
unexpected adverse effects on performance, since it allows one job to use CPUs allocated 
exclusively to another job.  For this reason, it may not be advisable to configure 
consumable resources without also configuring task affinity.  Note that task affinity 
can also be useful when select/linear (whole node allocation) is configured, to improve 
performance by restricting each task to a particular socket or other subset of CPU 
resources on a node.</p>
<br><br>
<a name="Section2"></a>
<h2>Getting Information about CPU usage by Jobs/Steps/Tasks</h2>
<p>There is no easy way to generate a comprehensive set of CPU management information 
for a job/step (allocation, distribution and binding). However, several 
commands/options provide limited information about CPU usage.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="3" cellspacing="0" width="100%">
		<tbody><tr>
			<td bgcolor="#e0e0e0" width="30%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Command/Option</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="70%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Information</b></font></p>
			</td>
		</tr>
		<tr>
			<td width="30%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
scontrol show job option: 
<font face="Courier New, monospace">--details</font></font></p>
			</td>
			<td width="70%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
This option provides a list of the nodes selected for the job and the CPU ids allocated to the job on each 
node. Note that the CPU ids reported by this command are SLURM abstract CPU ids, not Linux/hardware CPU ids 
(as reported by, for example, /proc/cpuinfo).
</font></p>
			</td>
		</tr>
		<tr>
			<td width="30%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
Linux command: <font face="Courier New, monospace">env</font></font></p>
			</td>
			<td width="70%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
Many SLURM environment variables provide information related to node and CPU usage:
<br><br>
<font face="Courier New, monospace">
SLURM_JOB_CPUS_PER_NODE<br>
SLURM_CPUS_PER_TASK<br>
SLURM_CPU_BIND<br>
SLURM_DISTRIBUTION<br>
SLURM_NODELIST<br>
SLURM_TASKS_PER_NODE<br>
SLURM_STEP_NODELIST<br>
SLURM_STEP_NUM_NODES<br>
SLURM_STEP_NUM_TASKS<br>
SLURM_STEP_TASKS_PER_NODE<br>
SLURM_NNODES<br>
SLURM_NTASKS<br>
SLURM_NPROCS<br>
SLURM_CPUS_ON_NODE<br>
SLURM_NODEID<br>
SLURMD_NODENAME<br>
</font>
</font></p>
			</td>
		</tr>
		<tr>
			<td width="30%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
srun/salloc/sbatch option: 
<font face="Courier New, monospace">--cpu_bind=verbose</font></font></p>
			</td>
			<td width="70%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
This option provides a list of the CPU masks used by task affinity to bind tasks to CPUs. 
Note that the CPU ids represented by these masks are Linux/hardware CPU ids, not SLURM 
abstract CPU ids as reported by scontrol, etc.
</font></p>
			</td>
		</tr>
		<tr>
			<td width="30%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
srun/salloc/sbatch option: 
<font face="Courier New, monospace">-l</font></font></p>
			</td>
			<td width="70%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
This option adds the task id as a prefix to each line of output from a task sent to stdout/stderr. 
This can be useful for distinguishing node-related and CPU-related information by task id 
for multi-task jobs/steps.
</font></p>
			</td>
		</tr>
		<tr>
			<td width="30%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
Linux command:<br> 
<font face="Courier New, monospace">cat /proc/&lt;pid&gt;/status | grep Cpus_allowed_list</font></font></p>
			</td>
			<td width="70%">
				<p align="LEFT"><font style="font-size: 8pt" size="1">
Given a task's pid (or "self" if the command is executed by the task itself), this command 
produces a list of the CPU ids bound to the task. This is the same information that is 
provided by <font face="Courier New, monospace">--cpu_bind=verbose</font>, but in a more readable format.
</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<h3>A Note on CPU Numbering</h3>
<p>The number and layout of logical CPUs known to SLURM is described in the node definitions in slurm.conf. This may
differ from the physical CPU layout on the actual hardware.  For this reason, SLURM generates its own internal, or
"abstract", CPU numbers.  These numbers may not match the physical, or "machine", CPU numbers known to Linux.
A utility to convert between SLURM abstract CPU numbers and physical machine CPU numbers is provided by SLURM.  See
module src/slurmd/slurmd/get_mach_stat.c for details.</p>
<br>
<a name="Section3"></a>
<h2>CPU Management and SLURM Accounting</h2>
<p>CPU management by SLURM users is subject to limits imposed by SLURM Accounting. Accounting limits may be applied on CPU
usage at the level of users, groups and clusters. For details, see the sacctmgr man page.</p>
<br>
<a name="Section4"></a>
<h2>CPU Management Examples</h2>
<p>The following examples illustrate some scenarios for managing CPU
resources using SLURM. Many additional scenarios are possible. In
each example, it is assumed that all CPUs on each node are available
for allocation.</p> 
<ul>
<li><a href="#Example">Example Node and Partition Configuration</a><br>
</li><li><a href="#Example1">Example 1: Allocation of whole nodes</a><br>
</li><li><a href="#Example2">Example 2: Simple allocation of cores as consumable resources</a>
</li><li><a href="#Example3">Example 3: Consumable resources with balanced allocation across nodes</a>
</li><li><a href="#Example4">Example 4: Consumable resources with minimization of resource fragmentation</a>
</li><li><a href="#Example5">Example 5: Consumable resources with cyclic distribution of tasks to nodes</a>
</li><li><a href="#Example6">Example 6: Consumable resources with default allocation and plane distribution of tasks to nodes</a>
</li><li><a href="#Example7">Example 7: Consumable resources with overcommitment of CPUs to tasks</a>
</li><li><a href="#Example8">Example 8: Consumable resources with resource sharing between jobs</a>
</li><li><a href="#Example9">Example 9: Consumable resources on multithreaded node, allocating only one thread per core</a>
</li><li><a href="#Example10">Example 10: Consumable resources with task affinity and core binding</a>
</li><li><a href="#Example11">Example 11: Consumable resources with task affinity and socket binding, Case 1</a>
</li><li><a href="#Example12">Example 12: Consumable resources with task affinity and socket binding, Case 2</a>
</li><li><a href="#Example13">Example 13: Consumable resources with task affinity and socket binding, Case 3</a>
</li><li><a href="#Example14">Example 14: Consumable resources with task affinity and customized allocation and distribution</a>
</li><li><a href="#Example15">Example 15: Consumable resources with task affinity to optimize the performance of a multi-task, 
multi-thread job</a>
</li><li><a href="#Example16">Example 16: Consumable resources with task cgroup and core binding</a>
</li></ul><br>
<a name="Example"></a>
<h3>Example Node and Partition Configuration</h3>
<p>For these examples, the SLURM cluster contains the following nodes:</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="402">
		<colgroup><col width="262">
		<col width="20">
		<col width="20">
		<col width="20">
		<col width="19">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="262">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n3</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="262">
				<p align="CENTER"><font style="font-size: 8pt" size="1">Number
				of Sockets</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="25" width="262">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1">Number
				of Cores per Socket</font></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="25" width="262">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1">Total
				Number of Cores</font></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="25" width="262">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1">Number
				of Threads (CPUs) per Core</font></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="24" width="262">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1">Total
				Number of CPUs</font></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">16</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<p>And the following partitions:</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="301">
		<colgroup><col width="109">
		<col width="72">
		<col width="82">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="109">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>PartitionName</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="72">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>regnodes</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="82">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>hypernode</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="109">
				<p align="CENTER"><font style="font-size: 8pt" size="1">Nodes</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="72">
				<p align="CENTER"><font style="font-size: 8pt" size="1">n0
				 n1  n2</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="82">
				<p align="CENTER"><font style="font-size: 8pt" size="1">n3</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="24" width="109">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1">Default</font></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="72">
				<p align="CENTER"><font style="font-size: 8pt" size="1">YES</font></p>
			</td>
			<td bgcolor="#e0e0e0" width="82">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<p>These entities are defined in slurm.conf as follows:</p>
<pre>Nodename=n0 NodeAddr=node0 Sockets=2 CoresPerSocket=4 ThreadsPerCore=1 Procs=8
Nodename=n1 NodeAddr=node1 Sockets=2 CoresPerSocket=4 ThreadsPerCore=1 Procs=8 State=IDLE
Nodename=n2 NodeAddr=node2 Sockets=2 CoresPerSocket=4 ThreadsPerCore=1 Procs=8 State=IDLE
Nodename=n3 NodeAddr=node3 Sockets=2 CoresPerSocket=4 ThreadsPerCore=2 Procs=16 State=IDLE
PartitionName=regnodes Nodes=n0,n1,n2 Shared=YES Default=YES State=UP 
PartitionName=hypernode Nodes=n3 State=UP
</pre>
<br>
<a name="Example1"></a>
<h3>Example 1: Allocation of whole nodes</h3>
<p>Allocate a minimum of two whole nodes to a job.</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/linear
</pre>
<p>Command line:</p>
<pre>srun --nodes=2 ...
</pre>
<p>Comments:</p>
The <font face="Courier New, monospace">SelectType=select/linear</font>
configuration option specifies allocation in units of whole nodes.
The<font face="Courier New, monospace"> --nodes=2</font> srun option causes
SLURM to allocate at least 2 nodes to the job.<p></p>
<br>
<a name="Example2"></a>
<h3>Example 2: Simple allocation of cores as consumable resources</h3>
<p>A job requires 6 CPUs (2 tasks and 3 CPUs per task with no overcommitment). Allocate the 6 CPUs as consumable resources 
from a single node in the default partition.</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --ntasks=2 --cpus-per-task=3 ...
</pre>
<p>Comments:</p>
<p>The <font face="Courier New, monospace">SelectType</font> configuration options define cores as consumable resources. 
The <font face="Courier New, monospace">--nodes=1-1</font> srun option
 restricts the job to a single node. The following table shows a possible pattern of allocation
  for this job.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="312">
		<colgroup><col width="204">
		<col width="20">
		<col width="20">
		<col width="19">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="204">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="25" width="204">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1"><b>Number
				of Allocated CPUs</b></font></font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="24" width="204">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1"><b>Number
				of Tasks</b></font></font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example3"></a>
<h3>Example 3: Consumable resources with balanced allocation across nodes</h3>
<p>A job requires 9 CPUs (3 tasks and 3 CPUs per task with no overcommitment). 
Allocate 3 CPUs from each of the 3 nodes in the default partition.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
</pre>
<p>Command line:</p>
<pre>srun --nodes=3-3 --ntasks=3 --cpus-per-task=3 ...
</pre>
<p>Comments:</p>
<p>The options specify the following conditions for the job: 3 tasks, 3 unique CPUs
 per task, using exactly 3 nodes. To satisfy these conditions, SLURM must
  allocate 3 CPUs from each node. The following table shows the allocation
   for this job.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="312">
		<colgroup><col width="204">
		<col width="20">
		<col width="20">
		<col width="19">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="204">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="25" width="204">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1"><b>Number
				of Allocated CPUs</b></font></font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="24" width="204">
				<p align="CENTER"><font color="#000000"><font style="font-size: 8pt" size="1"><b>Number
				of Tasks</b></font></font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example4"></a>
<h3>Example 4: Consumable resources with minimization of resource fragmentation</h3>
<p>A job requires 12 CPUs (12 tasks and 1 CPU per task with no overcommitment). Allocate 
CPUs using the minimum number of nodes and the minimum number of sockets required for 
the job in order to minimize fragmentation of allocated/unallocated CPUs in the cluster.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core,CR_CORE_DEFAULT_DIST_BLOCK
</pre>
<p>Command line:</p>
<pre>srun --ntasks=12 ...
</pre>
<p>Comments:</p>
<p>The default allocation method across nodes is block. This minimizes the number of nodes
 used for the job. The configuration option <font face="Courier New, monospace"> 
 CR_CORE_DEFAULT_DIST_BLOCK</font> sets the default allocation method within a 
 node to block. This minimizes the number of sockets used for the job within a node. 
 The combination of these two methods causes SLURM to allocate the 12 CPUs using the 
 minimum required number of nodes (2 nodes) and sockets (3 sockets).The following 
 table shows a possible pattern of allocation for this job.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="349">
		<colgroup><col width="204">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="9">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="204">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="32">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="32">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="31">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="204">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="204">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="204">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Tasks</b></font></p>
			</td>
			<td colspan="2" width="32">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td colspan="2" width="32">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="2" width="31">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example5"></a>
<h3>Example 5: Consumable resources with cyclic distribution of tasks to nodes</h3>
<p>A job requires 12 CPUs (6 tasks and 2 CPUs per task with no overcommitment). Allocate 
6 CPUs from each of 2 nodes in the default partition. Distribute tasks to nodes cyclically.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
</pre>
<p>Command line:</p>
<pre>srun --nodes=2-2 --ntasks-per-node=3 --distribution=cyclic <BR/>
--ntasks=6 --cpus-per-task=2 ...
</pre>
<p>Comments:</p>
<p>The options specify the following conditions for the job: 6 tasks, 2 unique CPUs per task, 
using exactly 2 nodes, and with 3 tasks per node. To satisfy these conditions, SLURM 
must allocate 6 CPUs from each of 2 nodes. The <font face="Courier New, monospace">
--distribution=cyclic</font> option causes the tasks to be distributed to the nodes in a 
round-robin fashion. The following table shows a possible pattern of allocation and 
distribution for this job.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="418">
		<colgroup><col width="310">
		<col width="20">
		<col width="20">
		<col width="19">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Tasks</b></font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Distribution
				of Tasks to Nodes, by Task id</b></font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
				0<br>2<br>4</font></p></td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">
				1<br>3<br>5</font></p></td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example6"></a>
<h3>Example 6: Consumable resources with default allocation and plane distribution of tasks to nodes</h3>
<p>A job requires 16 CPUs (8 tasks and 2 CPUs per task with no overcommitment). 
Use all 3 nodes in the default partition. Distribute tasks to each node in blocks of two in a round-robin fashion.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
</pre>
<p>Command line:</p>
<pre>srun --nodes=3-3 --distribution=plane=2 --ntasks=8 --cpus-per-task=2 ...
</pre>
<p>Comments:</p>
<p>The options specify the following conditions for the job: 8 tasks, 2 unique CPUs 
per task, using all 3 nodes in the partition. To satisfy these conditions using 
the default allocation method across nodes (block), SLURM allocates 8 CPUs from 
the first node, 6 CPUs from the second node and 2 CPUs from the third node. 
The <font face="Courier New, monospace">--distribution=plane=2</font> option causes SLURM 
to distribute tasks in blocks of two to each of the nodes in a round-robin fashion,
subject to the number of CPUs allocated on each node.  So, for example, only 1 task
is distributed to the third node because only 2 CPUs were allocated on that node and
each task requires 2 CPUs. The following table shows a possible pattern of allocation 
and distribution for this job.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="434">
		<colgroup><col width="310">
		<col width="28">
		<col width="28">
		<col width="19">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Tasks</b></font></p>
			</td>
			<td width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Distribution
				of Tasks to Nodes, by Task id</b></font></p>
			</td>
			<td width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0
				 1<br>5 6</font></p>
			</td>
			<td width="28">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2
				 3<br>7</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4<br> 
				</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example7"></a>
<h3>Example 7: Consumable resources with overcommitment of CPUs to tasks</h3>
<p>A job has 20 tasks. Run the job in a single node.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --ntasks=20 --overcommit ...
</pre>
<p>Comments:</p>
<p>The 
<font face="Courier New, monospace">--overcommit</font> option allows the job to 
run in only one node by overcommitting CPUs to tasks.The following table shows
 a possible pattern of allocation and distribution for this job.
</p>

<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="440">
		<colgroup><col width="310">
		<col width="42">
		<col width="20">
		<col width="19">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" height="17" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="42">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="18" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td width="42">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="18" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Tasks</b></font></p>
			</td>
			<td width="42">
				<p align="CENTER"><font style="font-size: 8pt" size="1">20</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" height="17" width="310">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Distribution
				of Tasks to Nodes, by Task id</b></font></p>
			</td>
			<td width="42">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0
				- 19</font></p>
			</td>
			<td width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example8"></a>
<h3>Example 8: Consumable resources with resource sharing between jobs</h3>
<p>2 jobs each require 6 CPUs (6 tasks per job with no overcommitment). 
Run both jobs simultaneously in a single node.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --nodelist=n0 --ntasks=6 --share ...
srun --nodes=1-1 --nodelist=n0 --ntasks=6 --share ...
</pre>
<p>Comments:</p>
<p>The <font face="Courier New, monospace">--nodes=1-1</font> and <font face="Courier New, monospace">-w n0</font> 
srun options together restrict both jobs to node n0. The 
<font face="Courier New, monospace">Shared=YES</font> option in the partition definition plus 
the <font face="Courier New, monospace">--share</font> srun option allows the two 
jobs to share CPUs on the node.
</p><br>
<a name="Example9"></a>
<h3>Example 9: Consumable resources on multithreaded node, allocating only one thread per core</h3>
<p>A job requires 8 CPUs (8 tasks with no overcommitment). Run the job on node n3, 
allocating only one thread per core.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_CPU
</pre>
<p>Command line:</p>
<pre>srun --partition=hypernode --ntasks=8 --hint=nomultithread ...
</pre>
<p>Comments:</p>
<p>The <font face="Courier New, monospace">CR_CPU</font> configuration 
option enables the allocation of only one thread per core. 
The <font face="Courier New, monospace">--hint=nomultithread</font> 
srun option causes SLURM to allocate only one thread from each core to 
this job. The following table shows a possible pattern of allocation 
for this job.
</p>
 <center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="3" cellspacing="0" width="591">
		<colgroup><col width="210">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="16">
		<col width="20">
		<col width="20">
		<col width="20">
		<col width="20">
		<col width="20">
		<col width="19">
		</colgroup><tbody><tr>
			<td bgcolor="#e0e0e0" width="210">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="16" bgcolor="#e0e0e0" width="367">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n3</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="210">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="170">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="191">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="210">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Core id</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="38">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="38">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="38">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="38">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="38">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="45">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="45">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td colspan="2" bgcolor="#e0e0e0" width="44">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="210">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>8</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="16">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>9</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>10</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>11</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>12</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>13</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="20">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>14</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="19">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>15</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="210">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="8" width="170">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="8" width="191">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="210">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="8" width="170">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0
				  2   4   6</font></p>
			</td>
			<td colspan="8" width="191">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8
				 10  12  14</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example10"></a>
<h3>Example 10: Consumable resources with task affinity and core binding</h3>
<p>A job requires 6 CPUs (6 tasks with no overcommitment). Run the job in a 
single node in the default partition. Apply core binding to each task.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
TaskPlugin=task/affinity
TaskPluginParam=sched
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --ntasks=6 --cpu_bind=cores ...
</pre>
<p>Comments:</p>
<p>Using the default allocation method within nodes (cyclic), SLURM allocates 
3 CPUs on each socket of 1 node. Using the default distribution method 
within nodes (cyclic), SLURM distributes and binds each task to an allocated 
core in a round-robin fashion across the sockets. The following table shows 
a possible pattern of allocation, distribution and binding for this job. 
For example, task id 2 is bound to CPU id 1.
</p><p>
</p><center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="443">
		<colgroup><col width="196">
		<col width="47">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="9">
		</colgroup><tbody><tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="163">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="4" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="4" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4 5 6</font></p>
			</td>
		</tr>
		<tr>
			<td rowspan="2" bgcolor="#e0e0e0" width="196">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Binding of
				Tasks to CPUs</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="47">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="47">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Task id</b></font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">5</font></p>
			</td>
			<td width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
</a><a name="Example11"></a>
<h3>Example 11: Consumable resources with task affinity and socket binding, Case 1</h3>
<p>A job requires 6 CPUs (6 tasks with no overcommitment). Run the job in 
a single node in the default partition. Apply socket binding to each task.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
TaskPlugin=task/affinity
TaskPluginParam=sched
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --ntasks=6 --cpu_bind=sockets ...
</pre>
<p>Comments:</p>
<p>Using the default allocation method within nodes (cyclic), SLURM allocates 3 
CPUs on each socket of 1 node. Using the default distribution method within nodes 
(cyclic), SLURM distributes and binds each task to all of the allocated CPUs in 
one socket in a round-robin fashion across the sockets. The following table shows 
a possible pattern of allocation, distribution and binding for this job. For 
example, task ids 1, 3 and 5 are all bound to CPU ids 4, 5 and 6.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="443">
		<colgroup><col width="196">
		<col width="47">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="9">
		</colgroup><tbody><tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="163">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="4" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="4" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4 5 6</font></p>
			</td>
		</tr>
		<tr>
			<td rowspan="2" bgcolor="#e0e0e0" width="196">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Binding of
				Tasks to CPUs</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="47">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="47">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Task ids</b></font></p>
			</td>
			<td colspan="3" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0
				 2  4</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td colspan="3" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1
				 3  5</font></p>
			</td>
			<td width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example12"></a>
<h3>Example 12: Consumable resources with task affinity and socket binding, Case 2</h3>
<p>A job requires 6 CPUs (2 tasks with 3 cpus per task and no overcommitment). Run the job in 
a single node in the default partition. Allocate cores using the block allocation method.
Distribute cores using the block distribution method. Apply socket binding to each task.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core,CR_CORE_DEFAULT_DIST_BLOCK
TaskPlugin=task/affinity
TaskPluginParam=sched
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --ntasks=2 --cpus-per-task=3 --cpu_bind=sockets <BR/>
--distribution=block:block ...
</pre>
<p>Comments:</p>
<p>Using the block allocation method, SLURM allocates 4 
CPUs on one socket and 2 CPUs on the other socket of one node. Using the block distribution method within  
nodes, SLURM distributes 3 CPUs to each task.  Applying socket binding, SLURM binds each task to all
allocated CPUs in all sockets in which the task has a distributed CPU. The following table shows 
a possible pattern of allocation, distribution and binding for this job. In this example, using the
block allocation method CPU ids 0-3 are allocated on socket id 0 and CPU ids 4-5 are allocated on 
socket id 1.  Using the block distribution method, CPU ids 0-2 were distributed to task id 0, and CPU ids 
3-5 were distributed to task id 1.  Applying socket binding, task id 0 is therefore bound to the allocated 
CPUs on socket 0, and task id 1 is bound to the allocated CPUs on both sockets.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="404">
		<colgroup><col width="196">
		<col width="47">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="9">
		</colgroup><tbody><tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="150">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2 3</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4 5</font></p>
			</td>
		</tr>
		<tr>
			<td rowspan="2" bgcolor="#e0e0e0" width="196">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Binding of
				Tasks to CPUs</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Task ids</b></font></p>
			</td>
			<td colspan="4" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1</font></p>
			</td>
			<td colspan="2" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td colspan="2" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example13"></a>
<h3>Example 13: Consumable resources with task affinity and socket binding, Case 3</h3>
<p>A job requires 6 CPUs (2 tasks with 3 cpus per task and no overcommitment). Run the job in 
a single node in the default partition. Allocate cores using the block allocation method.
Distribute cores using the cyclic distribution method. Apply socket binding to each task.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core,CR_CORE_DEFAULT_DIST_BLOCK
TaskPlugin=task/affinity
TaskPluginParam=sched
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --ntasks=2 --cpus-per-task=3 --cpu_bind=sockets <BR/>
--distribution=block:cyclic ...
</pre>
<p>Comments:</p>
<p>Using the block allocation method, SLURM allocates 4 
CPUs on one socket and 2 CPUs on the other socket of one node. Using the cyclic distribution method within  
nodes, SLURM distributes 3 CPUs to each task.  Applying socket binding, SLURM binds each task to all
allocated CPUs in all sockets in which the task has a distributed CPU. The following table shows 
a possible pattern of allocation, distribution and binding for this job. In this example, using the
block allocation method CPU ids 0-3 are allocated on socket id 0 and CPU ids 4-5 are allocated on 
socket id 1.  Using the cyclic distribution method, CPU ids 0, 1 and 4 were distributed to task id 0, and CPU ids 
2, 3 and 5 were distributed to task id 1.  Applying socket binding, both tasks are therefore bound to the 
allocated CPUs on both sockets.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="404">
		<colgroup><col width="196">
		<col width="47">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		</colgroup><tbody><tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="150">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2 3</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4 5</font></p>
			</td>
		</tr>
		<tr>
			<td rowspan="2" bgcolor="#e0e0e0" width="196">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Binding of
				Tasks to CPUs</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="58">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="18">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="58">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Task ids</b></font></p>
			</td>
			<td colspan="4" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1</font></p>
			</td>
			<td colspan="2" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1</font></p>
			</td>
			<td colspan="2" width="54">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example14"></a>
<h3>Example 14: Consumable resources with task affinity and customized allocation and distribution</h3>
<p>A job requires 18 CPUs (18 tasks with no overcommitment). Run the job in the 
default partition. Allocate 6 CPUs on each node using block allocation within 
nodes. Use cyclic distribution of tasks to nodes and block distribution of 
tasks for CPU binding.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core,CR_CORE_DEFAULT_DIST_BLOCK
TaskPlugin=task/affinity
TaskPluginParam=sched
</pre>
<p>Command line:</p>
<pre>srun --nodes=3-3 --ntasks=18 --ntasks-per-node=6 <BR/>
--distribution=cyclic:block --cpu_bind=cores ...
</pre>
<p>Comments:</p>
<p>This example shows the use of task affinity with customized allocation of CPUs and 
distribution of tasks across nodes and within nodes for binding. The srun options 
specify the following conditions for the job: 18 tasks, 1 unique CPU per task, using 
all 3 nodes in the partition, with 6 tasks per node. 
The <font face="Courier New, monospace">CR_CORE_DEFAULT_DIST_BLOCK</font> 
configuration option specifies block allocation within nodes. To satisfy these 
conditions, SLURM allocates 6 CPUs on each node, with 4 CPUs allocated on one socket 
and 2 CPUs on the other socket. The <font face="Courier New, monospace">
--distribution=cyclic:block</font> option specifies cyclic distribution of 
tasks to nodes and block distribution of tasks to CPUs within nodes for binding. 
The following table shows a possible pattern of allocation, distribution and binding 
for this job. For example, task id 10 is bound to CPU id 3 on node n1.
</p>
<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="1" cellspacing="0" width="100%">
		<tbody><tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2 3 4 5</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2 3 4 5</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2 3 4 5</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Tasks</b></font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Distribution
				of Tasks to Nodes, by Task id</b></font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0<br>
                        3<br>
                        6<br>
                        9<br>
                        12<br>
                        15
                        </font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1<br>
                        4<br>
                        7<br>
                        10<br>
                        13<br>
                        16
                        </font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2<br>
                        5<br>
                        8<br>
                        11<br>
                        14<br>
                        17
                         </font></p>
			</td>
		</tr>
		<tr>
			<td rowspan="2" bgcolor="#e0e0e0" width="15%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Binding of
				Tasks to CPUs</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="9%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="9%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Task id</b></font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">6</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">9</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">12</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">15</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">7</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">10</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">13</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">16</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">5</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">8</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">11</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">14</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">17</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example15"></a>
<h3>Example 15: Consumable resources with task affinity to optimize the performance of a multi-task, 
multi-thread job</h3>
<p>A job requires 9 CPUs (3 tasks and 3 CPUs per task with no overcommitment). Run 
the job in the default partition, managing the CPUs to optimize the performance 
of the job.</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core,CR_CORE_DEFAULT_DIST_BLOCK
TaskPlugin=task/affinity
TaskPluginParam=sched
</pre>
<p>Command line:</p>
<pre>srun --ntasks=3 --cpus-per-task=3 --ntasks-per-node=1 --cpu_bind=cores ...
</pre>
<p>Comments:</p>
<p>To optimize the performance of this job, the user wishes to allocate 3 CPUs from each of 
3 sockets and bind each task to the 3 CPUs in a single socket. The 
<font face="Courier New, monospace">SelectTypeParameters</font> configuration option specifies 
a consumable resource type of cores and block allocation within nodes. The 
<font face="Courier New, monospace">TaskPlugin</font> 
<font face="Courier New, monospace">and TaskPluginParam</font> 
configuration options enable task affinity. The srun options specify the following 
conditions for the job: 3 tasks, with 3 unique CPUs per task, with 1 task per node. To satisfy 
these conditions, SLURM allocates 3 CPUs from one socket in each of the 3 nodes in the default partition. The 
<font face="Courier New, monospace">--cpu_bind=cores</font> option causes SLURM to bind 
each task to the 3 allocated CPUs on the node to which it is distributed. The 
following table shows a possible pattern of allocation, distribution and binding 
for this job. For example, task id 2 is bound to CPU ids 0, 1 and 2 on socket id 0 of node n2.
</p>

<center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="1" cellspacing="0" width="100%">
		<tbody><tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n1</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n2</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td colspan="4" width="12%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Tasks</b></font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="28%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Distribution
				of Tasks to Nodes, by Task id</b></font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td colspan="8" width="24%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td rowspan="2" bgcolor="#e0e0e0" width="15%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Binding of
				Tasks to CPUs</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="9%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="3%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="9%">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Task id</b></font></p>
			</td>
			<td colspan="3" width="9%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td colspan="5" width="15%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td colspan="3" width="9%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td colspan="5" width="15%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td colspan="3" width="9%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td colspan="5" width="15%">
				<p align="CENTER"><font style="font-size: 8pt" size="1">--</font></p>
			</td>
		</tr>
	</tbody></table>
</center>
<br>
<a name="Example16"></a>
<h3>Example 16: Consumable resources with task cgroup and core binding</h3>
<p>A job requires 6 CPUs (6 tasks with no overcommitment). Run the job in a 
single node in the default partition. Apply core binding to each task using the task/cgroup plugin.
</p>
<p>slurm.conf options:</p>
<pre>SelectType=select/cons_res
SelectTypeParameters=CR_Core
TaskPlugin=task/cgroup
</pre>
<p>cgroup.conf options:</p>
<pre>ConstrainCores=yes
TaskAffinity=yes
</pre>
<p>Command line:</p>
<pre>srun --nodes=1-1 --ntasks=6 --cpu_bind=cores ...
</pre>
<p>Comments:</p>
<p>The task/cgroup plugin currently supports only the block method for
allocating cores within nodes and distributing tasks to CPUs for binding.
The following table shows a possible pattern of allocation, distribution 
and binding for this job. For example, task id 2 is bound to CPU id 2.
</p><p>
</p><center>
	<table style="page-break-inside: avoid; font-family: Arial,Helvetica,sans-serif;" border="1" bordercolor="#000000" cellpadding="6" cellspacing="0" width="443">
		<colgroup><col width="196">
		<col width="47">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="10">
		<col width="9">
		</colgroup><tbody><tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Nodename</b></font></p>
			</td>
			<td colspan="8" bgcolor="#e0e0e0" width="163">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>n0</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Socket id</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td colspan="4" bgcolor="#e0e0e0" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Number of
				Allocated CPUs</b></font></p>
			</td>
			<td colspan="4" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
		</tr>
		<tr>
			<td colspan="2" bgcolor="#e0e0e0" width="254">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Allocated
				CPU ids</b></font></p>
			</td>
			<td colspan="4" width="76">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0 1 2 3</font></p>
			</td>
			<td colspan="4" width="75">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4 5</font></p>
			</td>
		</tr>
		<tr>
			<td rowspan="2" bgcolor="#e0e0e0" width="196">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Binding of
				Tasks to CPUs</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="47">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>CPU id</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>0</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>1</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>2</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>3</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>4</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>5</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>6</b></font></p>
			</td>
			<td bgcolor="#e0e0e0" width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>7</b></font></p>
			</td>
		</tr>
		<tr>
			<td bgcolor="#e0e0e0" width="47">
				<p align="CENTER"><font style="font-size: 8pt" size="1"><b>Task id</b></font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">0</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">1</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">2</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">3</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">4</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">5</font></p>
			</td>
			<td width="10">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
			<td width="9">
				<p align="CENTER"><font style="font-size: 8pt" size="1">-</font></p>
			</td>
		</tr>
	</tbody></table>
</center>

<p class="footer"><a href="#top">top</a></p>

<p style="text-align:center;">Last modified 7 February 2012</p>

<!--#include virtual="footer.txt"-->