numactl.8 - libnuma-cloud-tpu - Git at Google

 .\" t
 .\" Copyright 2003,2004 Andi Kleen, SuSE Labs.
 .\"
 .\" Permission is granted to make and distribute verbatim copies of this
 .\" manual provided the copyright notice and this permission notice are
 .\" preserved on all copies.
 .\"
 .\" Permission is granted to copy and distribute modified versions of this
 .\" manual under the conditions for verbatim copying, provided that the
 .\" entire resulting derived work is distributed under the terms of a
 .\" permission notice identical to this one.
 .\"
 .\" Since the Linux kernel and libraries are constantly changing, this
 .\" manual page may be incorrect or out-of-date.  The author(s) assume no
 .\" responsibility for errors or omissions, or for damages resulting from
 .\" the use of the information contained herein.
 .\"
 .\" Formatted or processed versions of this manual, if unaccompanied by
 .\" the source, must acknowledge the copyright and authors of this work.
 .TH NUMACTL 8 "Mar 2004" "SuSE Labs" "Linux Administrator's Manual"
 .SH NAME
 numactl \- Control NUMA policy for processes or shared memory
 .SH SYNOPSIS
 .B numactl
 [
 .B \-\-all
 ] [
 .B \-\-interleave nodes
 ] [
 .B \-\-preferred node
 ] [
 .B \-\-membind nodes
 ] [
 .B \-\-cpunodebind nodes
 ] [
 .B \-\-physcpubind cpus
 ] [
 .B \-\-localalloc
 ] [\-\-] command {arguments ...}
 .br
 .B numactl \-\-show
 .br
 .B numactl \-\-hardware
 .br
 .B numactl
 [
 .B \-\-huge
 ] [
 .B \-\-offset offset
 ] [
 .B \-\-shmmode shmmode
 ] [
 .B \-\-length length
 ] [
 .B \-\-strict
 ]
 .br
 [
 .B \-\-shmid id
 ]
 .B \-\-shm shmkeyfile
 |
 .B \-\-file tmpfsfile
 .br
 [
 .B \-\-touch
 ] [
 .B \-\-dump
 ] [
 .B \-\-dump-nodes
 ]
 memory policy
 .SH DESCRIPTION
 .B numactl
 runs processes with a specific NUMA scheduling or memory placement policy.
 The policy is set for command and inherited by all of its children.
 In addition it can set persistent policy for shared memory segments or files.
 .PP
 Use -- before command if using command options that could be confused
 with numactl options.
 .PP
 .I nodes
 may be specified as N,N,N or  N-N or N,N-N or  N-N,N-N and so forth.
 Relative
 .I nodes
 may be specifed as +N,N,N or  +N-N or +N,N-N and so forth. The + indicates that
 the node numbers are relative to the process' set of allowed nodes in its
 current cpuset.
 A !N-N notation indicates the inverse of N-N, in other words all nodes
 except N-N.  If used with + notation, specify !+N-N. When
 .I same
 is specified the previous nodemask specified on the command line is used.
 all means all nodes in the current cpuset.
 .PP
 Instead of a number a node can also be:
 .TS
 tab(|);
 l l.
 netdev:DEV|The node connected to network device DEV.
 file:PATH |The node the block device of PATH.
 ip:HOST   |The node of the network device of HOST
 block:PATH|The node of block device PATH
 pci:[seg:]bus:dev[:func]|The node of a PCI device.
 .TE

 Note that block resolves the kernel block device names only
 for udev names in /dev use
 .I file:
 .TP
 Policy settings are:
 .TP
 .B \-\-all, \-a
 Unset default cpuset awareness, so user can use all possible CPUs/nodes
 for following policy settings.
 .TP
 .B \-\-interleave=nodes, \-i nodes
 Set a memory interleave policy. Memory will be allocated using round robin
 on
 .I nodes.
 When memory cannot be allocated on the current interleave target fall back
 to other nodes.
 Multiple nodes may be specified on --interleave, --membind and --cpunodebind.
 .TP
 .B \-\-membind=nodes, \-m nodes
 Only allocate memory from nodes.  Allocation will fail when there
 is not enough memory available on these nodes.
 .I nodes
 may be specified as noted above.
 .TP
 .B \-\-cpunodebind=nodes, \-N nodes
 Only execute
 .I command
 on the CPUs of
 .I nodes.
 Note that nodes may consist of multiple CPUs.
 .I nodes
 may be specified as noted above.
 .TP
 .B \-\-physcpubind=cpus, \-C cpus
 Only execute
 .I process
 on
 .I cpus.
 This accepts cpu numbers as shown in the
 .I processor
 fields of
 .I /proc/cpuinfo,
 or relative cpus as in relative to the current cpuset.
 You may specify "all", which means all cpus in the current cpuset.
 Physical
 .I cpus
 may be specified as N,N,N or  N-N or N,N-N or  N-N,N-N and so forth.
 Relative
 .I cpus
 may be specifed as +N,N,N or  +N-N or +N,N-N and so forth. The + indicates that
 the cpu numbers are relative to the process' set of allowed cpus in its
 current cpuset.
 A !N-N notation indicates the inverse of N-N, in other words all cpus
 except N-N.  If used with + notation, specify !+N-N.
 .TP
 .B \-\-localalloc, \-l
 Always allocate on the current node.
 .TP
 .B \-\-preferred=node
 Preferably allocate memory on
 .I node,
 but if memory cannot be allocated there fall back to other nodes.
 This option takes only a single node number.
 Relative notation may be used.
 .TP
 .B \-\-show, \-s
 Show NUMA policy settings of the current process.
 .TP
 .B \-\-hardware, \-H
 Show inventory of available nodes on the system.
 .TP 0
 Numactl can set up policy for a SYSV shared memory segment or a file in shmfs/hugetlbfs.

 This policy is persistent and will be used by
 all mappings from that shared memory. The order of options matters here.
 The specification must at least include either of
 .I \-\-shm,
 .I \-\-shmid,
 .I \-\-file
 to specify the shared memory segment or file and a memory policy like described
 above (
 .I \-\-interleave,
 .I \-\-localalloc,
 .I \-\-preferred,
 .I \-\-membind
 ).
 .TP
 .B \-\-huge
 When creating a SYSV shared memory segment use huge pages.
 Only valid before \-\-shmid or \-\-shm
 .TP
 .B \-\-offset
 Specify offset into the shared memory segment. Default 0.
 Valid units are
 .I m
 (for MB),
 .I g
 (for GB),
 .I k
 (for KB),
 otherwise it specifies bytes.
 .TP
 .B \-\-strict
 Give an error when a page in the policied area in the shared memory
 segment already was faulted in with a conflicting policy. Default
 is to silently ignore this.
 .TP
 .B \-\-shmmode shmmode
 Only valid before \-\-shmid or \-\-shm
 When creating a shared memory segment set it to numeric mode
 .I shmmode.
 .TP
 .B \-\-length length
 Apply policy to
 .I length
 range in the shared memory segment or make
 the segment length long
 Default is to use the remaining length
 Required when a shared memory segment is created and specifies the length
 of the new segment then. Valid units are
 .I m
 (for MB),
 .I g
 (for GB),
 .I k
 (for KB),
 otherwise it specifies bytes.
 .TP
 .B \-\-shmid id
 Create or use an shared memory segment with numeric ID
 .I id
 .TP
 .B \-\-shm shmkeyfile
 Create or use an shared memory segment, with the ID generated
 using
 .I ftok(3)
 from shmkeyfile
 .TP
 .B \-\-file tmpfsfile
 Set policy for a file in tmpfs or hugetlbfs
 .TP
 .B \-\-touch
 Touch pages to enforce policy early. Default is to not touch them, the policy
 is applied when an applications maps and accesses a page.
 .TP
 .B \-\-dump
 Dump policy in the specified range.
 .TP
 .B \-\-dump-nodes
 Dump all nodes of the specific range (very verbose!)
 .TP
 Valid node specifiers
 .TS
 tab(:);
 l l.
 all:All nodes
 number:Node number
 number1{,number2}:Node number1 and Node number2
 number1-number2:Nodes from number1 to number2
 ! nodes:Invert selection of the following specification.
 .TE
 .SH EXAMPLES
 numactl \-\-physcpubind=+0-4,8-12 myapplic arguments
 Run myapplic on cpus 0-4 and 8-12 of the current cpuset.

 numactl \-\-interleave=all bigdatabase arguments
 Run big database with its memory interleaved on all CPUs.

 numactl \-\-cpunodebind=0 \-\-membind=0,1 process
 Run process on node 0 with memory allocated on node 0 and 1.

 numactl \-\-cpunodebind=0 \-\-membind=0,1 -- process -l
 Run process as above, but with an option (-l) that would be confused with
 a numactl option.

 numactl \-\-cpunodebind=netdev:eth0 \-\-membind=netdev:eth0 network-server
 Run network-server on the node of network device eth0 with its memory
 also in the same node.

 numactl \-\-preferred=1 numactl \-\-show
 Set preferred node 1 and show the resulting state.

 numactl --interleave=all --shm /tmp/shmkey
 Interleave all of the sysv shared memory region specified by
 /tmp/shmkey over all nodes.

 Place a tmpfs file on 2 nodes:
   numactl --membind=2 dd if=/dev/zero of=/dev/shm/A bs=1M count=1024
   numactl --membind=3 dd if=/dev/zero of=/dev/shm/A seek=1024 bs=1M count=1024


 numactl --localalloc /dev/shm/file
 Reset the policy for the shared memory file
 .I file
 to the default localalloc policy.
 .SH NOTES
 Requires an NUMA policy aware kernel.

 Command is not executed using a shell. If you want to use shell metacharacters
 in the child use sh -c as wrapper.

 Setting policy for a hugetlbfs file does currently not work because
 it cannot be extended by truncate.

 Shared memory segments larger than numactl's address space cannot
 be completely policied. This could be a problem on 32bit architectures.
 Changing it piece by piece may work.

 The old
 .I --cpubind
 which accepts node numbers, not cpu numbers, is deprecated
 and replaced with the new
 .I --cpunodebind
 and
 .I --physcpubind
 options.

 .SH FILES
 .I /proc/cpuinfo
 for the listing of active CPUs. See
 .I proc(5)
 for details.

 .I /sys/devices/system/node/node*/numastat
 for NUMA memory hit statistics.

 .SH COPYRIGHT
 Copyright 2002,2004 Andi Kleen, SuSE Labs.
 numactl and the demo programs are under the GNU General Public License, v.2

 .SH SEE ALSO
 .I set_mempolicy(2)
 ,
 .I get_mempolicy(2)
 ,
 .I mbind(2)
 ,
 .I sched_setaffinity(2)
 ,
 .I sched_getaffinity(2)
 ,
 .I proc(5)
 ,
 .I ftok(3)
 ,
 .I shmat(2)
 ,
 .I migratepages(8)
	.\" t
	.\" Copyright 2003,2004 Andi Kleen, SuSE Labs.
	.\"
	.\" Permission is granted to make and distribute verbatim copies of this
	.\" manual provided the copyright notice and this permission notice are
	.\" preserved on all copies.
	.\"
	.\" Permission is granted to copy and distribute modified versions of this
	.\" manual under the conditions for verbatim copying, provided that the
	.\" entire resulting derived work is distributed under the terms of a
	.\" permission notice identical to this one.
	.\"
	.\" Since the Linux kernel and libraries are constantly changing, this
	.\" manual page may be incorrect or out-of-date. The author(s) assume no
	.\" responsibility for errors or omissions, or for damages resulting from
	.\" the use of the information contained herein.
	.\"
	.\" Formatted or processed versions of this manual, if unaccompanied by
	.\" the source, must acknowledge the copyright and authors of this work.
	.TH NUMACTL 8 "Mar 2004" "SuSE Labs" "Linux Administrator's Manual"
	.SH NAME
	numactl \- Control NUMA policy for processes or shared memory
	.SH SYNOPSIS
	.B numactl
	[
	.B \-\-all
	] [
	.B \-\-interleave nodes
	] [
	.B \-\-preferred node
	] [
	.B \-\-membind nodes
	] [
	.B \-\-cpunodebind nodes
	] [
	.B \-\-physcpubind cpus
	] [
	.B \-\-localalloc
	] [\-\-] command {arguments ...}
	.br
	.B numactl \-\-show
	.br
	.B numactl \-\-hardware
	.br
	.B numactl
	[
	.B \-\-huge
	] [
	.B \-\-offset offset
	] [
	.B \-\-shmmode shmmode
	] [
	.B \-\-length length
	] [
	.B \-\-strict
	]
	.br
	[
	.B \-\-shmid id
	]
	.B \-\-shm shmkeyfile
	\|
	.B \-\-file tmpfsfile
	.br
	[
	.B \-\-touch
	] [
	.B \-\-dump
	] [
	.B \-\-dump-nodes
	]
	memory policy
	.SH DESCRIPTION
	.B numactl
	runs processes with a specific NUMA scheduling or memory placement policy.
	The policy is set for command and inherited by all of its children.
	In addition it can set persistent policy for shared memory segments or files.
	.PP
	Use -- before command if using command options that could be confused
	with numactl options.
	.PP
	.I nodes
	may be specified as N,N,N or N-N or N,N-N or N-N,N-N and so forth.
	Relative
	.I nodes
	may be specifed as +N,N,N or +N-N or +N,N-N and so forth. The + indicates that
	the node numbers are relative to the process' set of allowed nodes in its
	current cpuset.
	A !N-N notation indicates the inverse of N-N, in other words all nodes
	except N-N. If used with + notation, specify !+N-N. When
	.I same
	is specified the previous nodemask specified on the command line is used.
	all means all nodes in the current cpuset.
	.PP
	Instead of a number a node can also be:
	.TS
	tab(\|);
	l l.
	netdev:DEV\|The node connected to network device DEV.
	file:PATH \|The node the block device of PATH.
	ip:HOST \|The node of the network device of HOST
	block:PATH\|The node of block device PATH
	pci:[seg:]bus:dev[:func]\|The node of a PCI device.
	.TE

	Note that block resolves the kernel block device names only
	for udev names in /dev use
	.I file:
	.TP
	Policy settings are:
	.TP
	.B \-\-all, \-a
	Unset default cpuset awareness, so user can use all possible CPUs/nodes
	for following policy settings.
	.TP
	.B \-\-interleave=nodes, \-i nodes
	Set a memory interleave policy. Memory will be allocated using round robin
	on
	.I nodes.
	When memory cannot be allocated on the current interleave target fall back
	to other nodes.
	Multiple nodes may be specified on --interleave, --membind and --cpunodebind.
	.TP
	.B \-\-membind=nodes, \-m nodes
	Only allocate memory from nodes. Allocation will fail when there
	is not enough memory available on these nodes.
	.I nodes
	may be specified as noted above.
	.TP
	.B \-\-cpunodebind=nodes, \-N nodes
	Only execute
	.I command
	on the CPUs of
	.I nodes.
	Note that nodes may consist of multiple CPUs.
	.I nodes
	may be specified as noted above.
	.TP
	.B \-\-physcpubind=cpus, \-C cpus
	Only execute
	.I process
	on
	.I cpus.
	This accepts cpu numbers as shown in the
	.I processor
	fields of
	.I /proc/cpuinfo,
	or relative cpus as in relative to the current cpuset.
	You may specify "all", which means all cpus in the current cpuset.
	Physical
	.I cpus
	may be specified as N,N,N or N-N or N,N-N or N-N,N-N and so forth.
	Relative
	.I cpus
	may be specifed as +N,N,N or +N-N or +N,N-N and so forth. The + indicates that
	the cpu numbers are relative to the process' set of allowed cpus in its
	current cpuset.
	A !N-N notation indicates the inverse of N-N, in other words all cpus
	except N-N. If used with + notation, specify !+N-N.
	.TP
	.B \-\-localalloc, \-l
	Always allocate on the current node.
	.TP
	.B \-\-preferred=node
	Preferably allocate memory on
	.I node,
	but if memory cannot be allocated there fall back to other nodes.
	This option takes only a single node number.
	Relative notation may be used.
	.TP
	.B \-\-show, \-s
	Show NUMA policy settings of the current process.
	.TP
	.B \-\-hardware, \-H
	Show inventory of available nodes on the system.
	.TP 0
	Numactl can set up policy for a SYSV shared memory segment or a file in shmfs/hugetlbfs.

	This policy is persistent and will be used by
	all mappings from that shared memory. The order of options matters here.
	The specification must at least include either of
	.I \-\-shm,
	.I \-\-shmid,
	.I \-\-file
	to specify the shared memory segment or file and a memory policy like described
	above (
	.I \-\-interleave,
	.I \-\-localalloc,
	.I \-\-preferred,
	.I \-\-membind
	).
	.TP
	.B \-\-huge
	When creating a SYSV shared memory segment use huge pages.
	Only valid before \-\-shmid or \-\-shm
	.TP
	.B \-\-offset
	Specify offset into the shared memory segment. Default 0.
	Valid units are
	.I m
	(for MB),
	.I g
	(for GB),
	.I k
	(for KB),
	otherwise it specifies bytes.
	.TP
	.B \-\-strict
	Give an error when a page in the policied area in the shared memory
	segment already was faulted in with a conflicting policy. Default
	is to silently ignore this.
	.TP
	.B \-\-shmmode shmmode
	Only valid before \-\-shmid or \-\-shm
	When creating a shared memory segment set it to numeric mode
	.I shmmode.
	.TP
	.B \-\-length length
	Apply policy to
	.I length
	range in the shared memory segment or make
	the segment length long
	Default is to use the remaining length
	Required when a shared memory segment is created and specifies the length
	of the new segment then. Valid units are
	.I m
	(for MB),
	.I g
	(for GB),
	.I k
	(for KB),
	otherwise it specifies bytes.
	.TP
	.B \-\-shmid id
	Create or use an shared memory segment with numeric ID
	.I id
	.TP
	.B \-\-shm shmkeyfile
	Create or use an shared memory segment, with the ID generated
	using
	.I ftok(3)
	from shmkeyfile
	.TP
	.B \-\-file tmpfsfile
	Set policy for a file in tmpfs or hugetlbfs
	.TP
	.B \-\-touch
	Touch pages to enforce policy early. Default is to not touch them, the policy
	is applied when an applications maps and accesses a page.
	.TP
	.B \-\-dump
	Dump policy in the specified range.
	.TP
	.B \-\-dump-nodes
	Dump all nodes of the specific range (very verbose!)
	.TP
	Valid node specifiers
	.TS
	tab(:);
	l l.
	all:All nodes
	number:Node number
	number1{,number2}:Node number1 and Node number2
	number1-number2:Nodes from number1 to number2
	! nodes:Invert selection of the following specification.
	.TE
	.SH EXAMPLES
	numactl \-\-physcpubind=+0-4,8-12 myapplic arguments
	Run myapplic on cpus 0-4 and 8-12 of the current cpuset.

	numactl \-\-interleave=all bigdatabase arguments
	Run big database with its memory interleaved on all CPUs.

	numactl \-\-cpunodebind=0 \-\-membind=0,1 process
	Run process on node 0 with memory allocated on node 0 and 1.

	numactl \-\-cpunodebind=0 \-\-membind=0,1 -- process -l
	Run process as above, but with an option (-l) that would be confused with
	a numactl option.

	numactl \-\-cpunodebind=netdev:eth0 \-\-membind=netdev:eth0 network-server
	Run network-server on the node of network device eth0 with its memory
	also in the same node.

	numactl \-\-preferred=1 numactl \-\-show
	Set preferred node 1 and show the resulting state.

	numactl --interleave=all --shm /tmp/shmkey
	Interleave all of the sysv shared memory region specified by
	/tmp/shmkey over all nodes.

	Place a tmpfs file on 2 nodes:
	numactl --membind=2 dd if=/dev/zero of=/dev/shm/A bs=1M count=1024
	numactl --membind=3 dd if=/dev/zero of=/dev/shm/A seek=1024 bs=1M count=1024


	numactl --localalloc /dev/shm/file
	Reset the policy for the shared memory file
	.I file
	to the default localalloc policy.
	.SH NOTES
	Requires an NUMA policy aware kernel.

	Command is not executed using a shell. If you want to use shell metacharacters
	in the child use sh -c as wrapper.

	Setting policy for a hugetlbfs file does currently not work because
	it cannot be extended by truncate.

	Shared memory segments larger than numactl's address space cannot
	be completely policied. This could be a problem on 32bit architectures.
	Changing it piece by piece may work.

	The old
	.I --cpubind
	which accepts node numbers, not cpu numbers, is deprecated
	and replaced with the new
	.I --cpunodebind
	and
	.I --physcpubind
	options.

	.SH FILES
	.I /proc/cpuinfo
	for the listing of active CPUs. See
	.I proc(5)
	for details.

	.I /sys/devices/system/node/node*/numastat
	for NUMA memory hit statistics.

	.SH COPYRIGHT
	Copyright 2002,2004 Andi Kleen, SuSE Labs.
	numactl and the demo programs are under the GNU General Public License, v.2

	.SH SEE ALSO
	.I set_mempolicy(2)
	,
	.I get_mempolicy(2)
	,
	.I mbind(2)
	,
	.I sched_setaffinity(2)
	,
	.I sched_getaffinity(2)
	,
	.I proc(5)
	,
	.I ftok(3)
	,
	.I shmat(2)
	,
	.I migratepages(8)