2050 lines
52 KiB
HTML
2050 lines
52 KiB
HTML
|
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<HTML><HEAD><TITLE>Man page of CPUSET</TITLE>
|
|
</HEAD><BODY>
|
|
<H1>CPUSET</H1>
|
|
Section: Linux Programmer's Manual (7)<BR>Updated: 2017-09-15<BR><A HREF="#index">Index</A>
|
|
<A HREF="/cgi-bin/man/man2html">Return to Main Contents</A><HR>
|
|
|
|
<A NAME="lbAB"> </A>
|
|
<H2>NAME</H2>
|
|
|
|
cpuset - confine processes to processor and memory node subsets
|
|
<A NAME="lbAC"> </A>
|
|
<H2>DESCRIPTION</H2>
|
|
|
|
The cpuset filesystem is a pseudo-filesystem interface
|
|
to the kernel cpuset mechanism,
|
|
which is used to control the processor placement
|
|
and memory placement of processes.
|
|
It is commonly mounted at
|
|
<I>/dev/cpuset</I>.
|
|
|
|
<P>
|
|
|
|
On systems with kernels compiled with built in support for cpusets,
|
|
all processes are attached to a cpuset, and cpusets are always present.
|
|
If a system supports cpusets, then it will have the entry
|
|
<B>nodev cpuset</B>
|
|
|
|
in the file
|
|
<I>/proc/filesystems</I>.
|
|
|
|
By mounting the cpuset filesystem (see the
|
|
<B>EXAMPLE</B>
|
|
|
|
section below),
|
|
the administrator can configure the cpusets on a system
|
|
to control the processor and memory placement of processes
|
|
on that system.
|
|
By default, if the cpuset configuration
|
|
on a system is not modified or if the cpuset filesystem
|
|
is not even mounted, then the cpuset mechanism,
|
|
though present, has no effect on the system's behavior.
|
|
<P>
|
|
|
|
A cpuset defines a list of CPUs and memory nodes.
|
|
<P>
|
|
|
|
The CPUs of a system include all the logical processing
|
|
units on which a process can execute, including, if present,
|
|
multiple processor cores within a package and Hyper-Threads
|
|
within a processor core.
|
|
Memory nodes include all distinct
|
|
banks of main memory; small and SMP systems typically have
|
|
just one memory node that contains all the system's main memory,
|
|
while NUMA (non-uniform memory access) systems have multiple memory nodes.
|
|
<P>
|
|
|
|
Cpusets are represented as directories in a hierarchical
|
|
pseudo-filesystem, where the top directory in the hierarchy
|
|
(<I>/dev/cpuset</I>)
|
|
|
|
represents the entire system (all online CPUs and memory nodes)
|
|
and any cpuset that is the child (descendant) of
|
|
another parent cpuset contains a subset of that parent's
|
|
CPUs and memory nodes.
|
|
The directories and files representing cpusets have normal
|
|
filesystem permissions.
|
|
<P>
|
|
|
|
Every process in the system belongs to exactly one cpuset.
|
|
A process is confined to run only on the CPUs in
|
|
the cpuset it belongs to, and to allocate memory only
|
|
on the memory nodes in that cpuset.
|
|
When a process
|
|
<B><A HREF="/cgi-bin/man/man2html?2+fork">fork</A></B>(2)s,
|
|
|
|
the child process is placed in the same cpuset as its parent.
|
|
With sufficient privilege, a process may be moved from one
|
|
cpuset to another and the allowed CPUs and memory nodes
|
|
of an existing cpuset may be changed.
|
|
<P>
|
|
|
|
When the system begins booting, a single cpuset is
|
|
defined that includes all CPUs and memory nodes on the
|
|
system, and all processes are in that cpuset.
|
|
During the boot process, or later during normal system operation,
|
|
other cpusets may be created, as subdirectories of this top cpuset,
|
|
under the control of the system administrator,
|
|
and processes may be placed in these other cpusets.
|
|
<P>
|
|
|
|
Cpusets are integrated with the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+sched_setaffinity">sched_setaffinity</A></B>(2)
|
|
|
|
scheduling affinity mechanism and the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mbind">mbind</A></B>(2)
|
|
|
|
and
|
|
<B><A HREF="/cgi-bin/man/man2html?2+set_mempolicy">set_mempolicy</A></B>(2)
|
|
|
|
memory-placement mechanisms in the kernel.
|
|
Neither of these mechanisms let a process make use
|
|
of a CPU or memory node that is not allowed by that process's cpuset.
|
|
If changes to a process's cpuset placement conflict with these
|
|
other mechanisms, then cpuset placement is enforced
|
|
even if it means overriding these other mechanisms.
|
|
The kernel accomplishes this overriding by silently
|
|
restricting the CPUs and memory nodes requested by
|
|
these other mechanisms to those allowed by the
|
|
invoking process's cpuset.
|
|
This can result in these
|
|
other calls returning an error, if for example, such
|
|
a call ends up requesting an empty set of CPUs or
|
|
memory nodes, after that request is restricted to
|
|
the invoking process's cpuset.
|
|
<P>
|
|
|
|
Typically, a cpuset is used to manage
|
|
the CPU and memory-node confinement for a set of
|
|
cooperating processes such as a batch scheduler job, and these
|
|
other mechanisms are used to manage the placement of
|
|
individual processes or memory regions within that set or job.
|
|
<A NAME="lbAD"> </A>
|
|
<H2>FILES</H2>
|
|
|
|
Each directory below
|
|
<I>/dev/cpuset</I>
|
|
|
|
represents a cpuset and contains a fixed set of pseudo-files
|
|
describing the state of that cpuset.
|
|
<P>
|
|
|
|
New cpusets are created using the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mkdir">mkdir</A></B>(2)
|
|
|
|
system call or the
|
|
<B><A HREF="/cgi-bin/man/man2html?1+mkdir">mkdir</A></B>(1)
|
|
|
|
command.
|
|
The properties of a cpuset, such as its flags, allowed
|
|
CPUs and memory nodes, and attached processes, are queried and modified
|
|
by reading or writing to the appropriate file in that cpuset's directory,
|
|
as listed below.
|
|
<P>
|
|
|
|
The pseudo-files in each cpuset directory are automatically created when
|
|
the cpuset is created, as a result of the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mkdir">mkdir</A></B>(2)
|
|
|
|
invocation.
|
|
It is not possible to directly add or remove these pseudo-files.
|
|
<P>
|
|
|
|
A cpuset directory that contains no child cpuset directories,
|
|
and has no attached processes, can be removed using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+rmdir">rmdir</A></B>(2)
|
|
|
|
or
|
|
<B><A HREF="/cgi-bin/man/man2html?1+rmdir">rmdir</A></B>(1).
|
|
|
|
It is not necessary, or possible,
|
|
to remove the pseudo-files inside the directory before removing it.
|
|
<P>
|
|
|
|
The pseudo-files in each cpuset directory are
|
|
small text files that may be read and
|
|
written using traditional shell utilities such as
|
|
<B><A HREF="/cgi-bin/man/man2html?1+cat">cat</A></B>(1),
|
|
|
|
and
|
|
<B><A HREF="/cgi-bin/man/man2html?1+echo">echo</A></B>(1),
|
|
|
|
or from a program by using file I/O library functions or system calls,
|
|
such as
|
|
<B><A HREF="/cgi-bin/man/man2html?2+open">open</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+read">read</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2),
|
|
|
|
and
|
|
<B><A HREF="/cgi-bin/man/man2html?2+close">close</A></B>(2).
|
|
|
|
<P>
|
|
|
|
The pseudo-files in a cpuset directory represent internal kernel
|
|
state and do not have any persistent image on disk.
|
|
Each of these per-cpuset files is listed and described below.
|
|
|
|
<DL COMPACT>
|
|
<DT id="1"><I>tasks</I>
|
|
|
|
<DD>
|
|
List of the process IDs (PIDs) of the processes in that cpuset.
|
|
The list is formatted as a series of ASCII
|
|
decimal numbers, each followed by a newline.
|
|
A process may be added to a cpuset (automatically removing
|
|
it from the cpuset that previously contained it) by writing its
|
|
PID to that cpuset's
|
|
<I>tasks</I>
|
|
|
|
file (with or without a trailing newline).
|
|
<DT id="2"><DD>
|
|
<B>Warning:</B>
|
|
|
|
only one PID may be written to the
|
|
<I>tasks</I>
|
|
|
|
file at a time.
|
|
If a string is written that contains more
|
|
than one PID, only the first one will be used.
|
|
|
|
<DT id="3"><I>notify_on_release</I>
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1), that cpuset will receive special handling
|
|
after it is released, that is, after all processes cease using
|
|
it (i.e., terminate or are moved to a different cpuset)
|
|
and all child cpuset directories have been removed.
|
|
See the <B>Notify On Release</B> section, below.
|
|
|
|
<DT id="4"><I>cpuset.cpus</I>
|
|
|
|
<DD>
|
|
List of the physical numbers of the CPUs on which processes
|
|
in that cpuset are allowed to execute.
|
|
See <B>List Format</B> below for a description of the
|
|
format of
|
|
<I>cpus</I>.
|
|
|
|
<DT id="5"><DD>
|
|
The CPUs allowed to a cpuset may be changed by
|
|
writing a new list to its
|
|
<I>cpus</I>
|
|
|
|
file.
|
|
|
|
<DT id="6"><I>cpuset.cpu_exclusive</I>
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1), the cpuset has exclusive use of
|
|
its CPUs (no sibling or cousin cpuset may overlap CPUs).
|
|
By default, this is off (0).
|
|
Newly created cpusets also initially default this to off (0).
|
|
<DT id="7"><DD>
|
|
Two cpusets are
|
|
<I>sibling</I>
|
|
|
|
cpusets if they share the same parent cpuset in the
|
|
<I>/dev/cpuset</I>
|
|
|
|
hierarchy.
|
|
Two cpusets are
|
|
<I>cousin</I>
|
|
|
|
cpusets if neither is the ancestor of the other.
|
|
Regardless of the
|
|
<I>cpu_exclusive</I>
|
|
|
|
setting, if one cpuset is the ancestor of another,
|
|
and if both of these cpusets have nonempty
|
|
<I>cpus</I>,
|
|
|
|
then their
|
|
<I>cpus</I>
|
|
|
|
must overlap, because the
|
|
<I>cpus</I>
|
|
|
|
of any cpuset are always a subset of the
|
|
<I>cpus</I>
|
|
|
|
of its parent cpuset.
|
|
|
|
<DT id="8"><I>cpuset.mems</I>
|
|
|
|
<DD>
|
|
List of memory nodes on which processes in this cpuset are
|
|
allowed to allocate memory.
|
|
See <B>List Format</B> below for a description of the
|
|
format of
|
|
<I>mems</I>.
|
|
|
|
|
|
<DT id="9"><I>cpuset.mem_exclusive</I>
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1), the cpuset has exclusive use of
|
|
its memory nodes (no sibling or cousin may overlap).
|
|
Also if set (1), the cpuset is a <B>Hardwall</B> cpuset (see below).
|
|
By default, this is off (0).
|
|
Newly created cpusets also initially default this to off (0).
|
|
<DT id="10"><DD>
|
|
Regardless of the
|
|
<I>mem_exclusive</I>
|
|
|
|
setting, if one cpuset is the ancestor of another,
|
|
then their memory nodes must overlap, because the memory
|
|
nodes of any cpuset are always a subset of the memory nodes
|
|
of that cpuset's parent cpuset.
|
|
|
|
<DT id="11"><I>cpuset.mem_hardwall</I> (since Linux 2.6.26)
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1), the cpuset is a <B>Hardwall</B> cpuset (see below).
|
|
Unlike <B>mem_exclusive</B>,
|
|
there is no constraint on whether cpusets
|
|
marked <B>mem_hardwall</B> may have overlapping
|
|
memory nodes with sibling or cousin cpusets.
|
|
By default, this is off (0).
|
|
Newly created cpusets also initially default this to off (0).
|
|
|
|
<DT id="12"><I>cpuset.memory_migrate</I> (since Linux 2.6.16)
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1), then memory migration is enabled.
|
|
By default, this is off (0).
|
|
See the <B>Memory Migration</B> section, below.
|
|
|
|
<DT id="13"><I>cpuset.memory_pressure</I> (since Linux 2.6.16)
|
|
|
|
<DD>
|
|
A measure of how much memory pressure the processes in this
|
|
cpuset are causing.
|
|
See the <B>Memory Pressure</B> section, below.
|
|
Unless
|
|
<I>memory_pressure_enabled</I>
|
|
|
|
is enabled, always has value zero (0).
|
|
This file is read-only.
|
|
See the
|
|
<B>WARNINGS</B>
|
|
|
|
section, below.
|
|
|
|
<DT id="14"><I>cpuset.memory_pressure_enabled</I> (since Linux 2.6.16)
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
This file is present only in the root cpuset, normally
|
|
<I>/dev/cpuset</I>.
|
|
|
|
If set (1), the
|
|
<I>memory_pressure</I>
|
|
|
|
calculations are enabled for all cpusets in the system.
|
|
By default, this is off (0).
|
|
See the
|
|
<B>Memory Pressure</B> section, below.
|
|
|
|
<DT id="15"><I>cpuset.memory_spread_page</I> (since Linux 2.6.17)
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1), pages in the kernel page cache
|
|
(filesystem buffers) are uniformly spread across the cpuset.
|
|
By default, this is off (0) in the top cpuset,
|
|
and inherited from the parent cpuset in
|
|
newly created cpusets.
|
|
See the <B>Memory Spread</B> section, below.
|
|
|
|
<DT id="16"><I>cpuset.memory_spread_slab</I> (since Linux 2.6.17)
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1), the kernel slab caches
|
|
for file I/O (directory and inode structures) are
|
|
uniformly spread across the cpuset.
|
|
By defaultBy default, is off (0) in the top cpuset,
|
|
and inherited from the parent cpuset in
|
|
newly created cpusets.
|
|
See the <B>Memory Spread</B> section, below.
|
|
|
|
<DT id="17"><I>cpuset.sched_load_balance</I> (since Linux 2.6.24)
|
|
|
|
<DD>
|
|
Flag (0 or 1).
|
|
If set (1, the default) the kernel will
|
|
automatically load balance processes in that cpuset over
|
|
the allowed CPUs in that cpuset.
|
|
If cleared (0) the
|
|
kernel will avoid load balancing processes in this cpuset,
|
|
<I>unless</I>
|
|
|
|
some other cpuset with overlapping CPUs has its
|
|
<I>sched_load_balance</I>
|
|
|
|
flag set.
|
|
See <B>Scheduler Load Balancing</B>, below, for further details.
|
|
|
|
<DT id="18"><I>cpuset.sched_relax_domain_level</I> (since Linux 2.6.26)
|
|
|
|
<DD>
|
|
Integer, between -1 and a small positive value.
|
|
The
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
controls the width of the range of CPUs over which the kernel scheduler
|
|
performs immediate rebalancing of runnable tasks across CPUs.
|
|
If
|
|
<I>sched_load_balance</I>
|
|
|
|
is disabled, then the setting of
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
does not matter, as no such load balancing is done.
|
|
If
|
|
<I>sched_load_balance</I>
|
|
|
|
is enabled, then the higher the value of the
|
|
<I>sched_relax_domain_level</I>,
|
|
|
|
the wider
|
|
the range of CPUs over which immediate load balancing is attempted.
|
|
See <B>Scheduler Relax Domain Level</B>, below, for further details.
|
|
|
|
</DL>
|
|
<P>
|
|
|
|
In addition to the above pseudo-files in each directory below
|
|
<I>/dev/cpuset</I>,
|
|
|
|
each process has a pseudo-file,
|
|
<I>/proc/<pid>/cpuset</I>,
|
|
|
|
that displays the path of the process's cpuset directory
|
|
relative to the root of the cpuset filesystem.
|
|
|
|
<P>
|
|
|
|
Also the
|
|
<I>/proc/<pid>/status</I>
|
|
|
|
file for each process has four added lines,
|
|
displaying the process's
|
|
<I>Cpus_allowed</I>
|
|
|
|
(on which CPUs it may be scheduled) and
|
|
<I>Mems_allowed</I>
|
|
|
|
(on which memory nodes it may obtain memory),
|
|
in the two formats <B>Mask Format</B> and <B>List Format</B> (see below)
|
|
as shown in the following example:
|
|
<P>
|
|
|
|
|
|
|
|
Cpus_allowed: ffffffff,ffffffff,ffffffff,ffffffff
|
|
Cpus_allowed_list: 0-127
|
|
Mems_allowed: ffffffff,ffffffff
|
|
Mems_allowed_list: 0-63
|
|
|
|
|
|
<P>
|
|
|
|
The "allowed" fields were added in Linux 2.6.24;
|
|
the "allowed_list" fields were added in Linux 2.6.26.
|
|
|
|
<A NAME="lbAE"> </A>
|
|
<H2>EXTENDED CAPABILITIES</H2>
|
|
|
|
In addition to controlling which
|
|
<I>cpus</I>
|
|
|
|
and
|
|
<I>mems</I>
|
|
|
|
a process is allowed to use, cpusets provide the following
|
|
extended capabilities.
|
|
|
|
<A NAME="lbAF"> </A>
|
|
<H3>Exclusive cpusets</H3>
|
|
|
|
If a cpuset is marked
|
|
<I>cpu_exclusive</I>
|
|
|
|
or
|
|
<I>mem_exclusive</I>,
|
|
|
|
no other cpuset, other than a direct ancestor or descendant,
|
|
may share any of the same CPUs or memory nodes.
|
|
<P>
|
|
|
|
A cpuset that is
|
|
<I>mem_exclusive</I>
|
|
|
|
restricts kernel allocations for
|
|
buffer cache pages and other internal kernel data pages
|
|
commonly shared by the kernel across
|
|
multiple users.
|
|
All cpusets, whether
|
|
<I>mem_exclusive</I>
|
|
|
|
or not, restrict allocations of memory for user space.
|
|
This enables configuring a
|
|
system so that several independent jobs can share common kernel data,
|
|
while isolating each job's user allocation in
|
|
its own cpuset.
|
|
To do this, construct a large
|
|
<I>mem_exclusive</I>
|
|
|
|
cpuset to hold all the jobs, and construct child,
|
|
non-<I>mem_exclusive</I>
|
|
|
|
cpusets for each individual job.
|
|
Only a small amount of kernel memory,
|
|
such as requests from interrupt handlers, is allowed to be
|
|
placed on memory nodes
|
|
outside even a
|
|
<I>mem_exclusive</I>
|
|
|
|
cpuset.
|
|
|
|
<A NAME="lbAG"> </A>
|
|
<H3>Hardwall</H3>
|
|
|
|
A cpuset that has
|
|
<I>mem_exclusive</I>
|
|
|
|
or
|
|
<I>mem_hardwall</I>
|
|
|
|
set is a
|
|
<I>hardwall</I>
|
|
|
|
cpuset.
|
|
A
|
|
<I>hardwall</I>
|
|
|
|
cpuset restricts kernel allocations for page, buffer,
|
|
and other data commonly shared by the kernel across multiple users.
|
|
All cpusets, whether
|
|
<I>hardwall</I>
|
|
|
|
or not, restrict allocations of memory for user space.
|
|
<P>
|
|
|
|
This enables configuring a system so that several independent
|
|
jobs can share common kernel data, such as filesystem pages,
|
|
while isolating each job's user allocation in its own cpuset.
|
|
To do this, construct a large
|
|
<I>hardwall</I>
|
|
|
|
cpuset to hold
|
|
all the jobs, and construct child cpusets for each individual
|
|
job which are not
|
|
<I>hardwall</I>
|
|
|
|
cpusets.
|
|
<P>
|
|
|
|
Only a small amount of kernel memory, such as requests from
|
|
interrupt handlers, is allowed to be taken outside even a
|
|
<I>hardwall</I>
|
|
|
|
cpuset.
|
|
|
|
<A NAME="lbAH"> </A>
|
|
<H3>Notify on release</H3>
|
|
|
|
If the
|
|
<I>notify_on_release</I>
|
|
|
|
flag is enabled (1) in a cpuset,
|
|
then whenever the last process in the cpuset leaves
|
|
(exits or attaches to some other cpuset)
|
|
and the last child cpuset of that cpuset is removed,
|
|
the kernel will run the command
|
|
<I>/sbin/cpuset_release_agent</I>,
|
|
|
|
supplying the pathname (relative to the mount point of the
|
|
cpuset filesystem) of the abandoned cpuset.
|
|
This enables automatic removal of abandoned cpusets.
|
|
<P>
|
|
|
|
The default value of
|
|
<I>notify_on_release</I>
|
|
|
|
in the root cpuset at system boot is disabled (0).
|
|
The default value of other cpusets at creation
|
|
is the current value of their parent's
|
|
<I>notify_on_release</I>
|
|
|
|
setting.
|
|
<P>
|
|
|
|
The command
|
|
<I>/sbin/cpuset_release_agent</I>
|
|
|
|
is invoked, with the name
|
|
(<I>/dev/cpuset</I>
|
|
|
|
relative path)
|
|
of the to-be-released cpuset in
|
|
<I>argv[1]</I>.
|
|
|
|
<P>
|
|
|
|
The usual contents of the command
|
|
<I>/sbin/cpuset_release_agent</I>
|
|
|
|
is simply the shell script:
|
|
<P>
|
|
|
|
|
|
|
|
#!/bin/sh
|
|
rmdir /dev/cpuset/$1
|
|
|
|
|
|
<P>
|
|
|
|
As with other flag values below, this flag can
|
|
be changed by writing an ASCII
|
|
number 0 or 1 (with optional trailing newline)
|
|
into the file, to clear or set the flag, respectively.
|
|
|
|
<A NAME="lbAI"> </A>
|
|
<H3>Memory pressure</H3>
|
|
|
|
The
|
|
<I>memory_pressure</I>
|
|
|
|
of a cpuset provides a simple per-cpuset running average of
|
|
the rate that the processes in a cpuset are attempting to free up in-use
|
|
memory on the nodes of the cpuset to satisfy additional memory requests.
|
|
<P>
|
|
|
|
This enables batch managers that are monitoring jobs running in dedicated
|
|
cpusets to efficiently detect what level of memory pressure that job
|
|
is causing.
|
|
<P>
|
|
|
|
This is useful both on tightly managed systems running a wide mix of
|
|
submitted jobs, which may choose to terminate or reprioritize jobs that
|
|
are trying to use more memory than allowed on the nodes assigned them,
|
|
and with tightly coupled, long-running, massively parallel scientific
|
|
computing jobs that will dramatically fail to meet required performance
|
|
goals if they start to use more memory than allowed to them.
|
|
<P>
|
|
|
|
This mechanism provides a very economical way for the batch manager
|
|
to monitor a cpuset for signs of memory pressure.
|
|
It's up to the batch manager or other user code to decide
|
|
what action to take if it detects signs of memory pressure.
|
|
<P>
|
|
|
|
Unless memory pressure calculation is enabled by setting the pseudo-file
|
|
<I>/dev/cpuset/cpuset.memory_pressure_enabled</I>,
|
|
|
|
it is not computed for any cpuset, and reads from any
|
|
<I>memory_pressure</I>
|
|
|
|
always return zero, as represented by the ASCII string "0\n".
|
|
See the <B>WARNINGS</B> section, below.
|
|
<P>
|
|
|
|
A per-cpuset, running average is employed for the following reasons:
|
|
<DL COMPACT>
|
|
<DT id="19">*<DD>
|
|
Because this meter is per-cpuset rather than per-process or per virtual
|
|
memory region, the system load imposed by a batch scheduler monitoring
|
|
this metric is sharply reduced on large systems, because a scan of
|
|
the tasklist can be avoided on each set of queries.
|
|
<DT id="20">*<DD>
|
|
Because this meter is a running average rather than an accumulating
|
|
counter, a batch scheduler can detect memory pressure with a
|
|
single read, instead of having to read and accumulate results
|
|
for a period of time.
|
|
<DT id="21">*<DD>
|
|
Because this meter is per-cpuset rather than per-process,
|
|
the batch scheduler can obtain the key information---memory
|
|
pressure in a cpuset---with a single read, rather than having to
|
|
query and accumulate results over all the (dynamically changing)
|
|
set of processes in the cpuset.
|
|
</DL>
|
|
<P>
|
|
|
|
The
|
|
<I>memory_pressure</I>
|
|
|
|
of a cpuset is calculated using a per-cpuset simple digital filter
|
|
that is kept within the kernel.
|
|
For each cpuset, this filter tracks
|
|
the recent rate at which processes attached to that cpuset enter the
|
|
kernel direct reclaim code.
|
|
<P>
|
|
|
|
The kernel direct reclaim code is entered whenever a process has to
|
|
satisfy a memory page request by first finding some other page to
|
|
repurpose, due to lack of any readily available already free pages.
|
|
Dirty filesystem pages are repurposed by first writing them
|
|
to disk.
|
|
Unmodified filesystem buffer pages are repurposed
|
|
by simply dropping them, though if that page is needed again, it
|
|
will have to be reread from disk.
|
|
<P>
|
|
|
|
The
|
|
<I>cpuset.memory_pressure</I>
|
|
|
|
file provides an integer number representing the recent (half-life of
|
|
10 seconds) rate of entries to the direct reclaim code caused by any
|
|
process in the cpuset, in units of reclaims attempted per second,
|
|
times 1000.
|
|
|
|
<A NAME="lbAJ"> </A>
|
|
<H3>Memory spread</H3>
|
|
|
|
There are two Boolean flag files per cpuset that control where the
|
|
kernel allocates pages for the filesystem buffers and related
|
|
in-kernel data structures.
|
|
They are called
|
|
<I>cpuset.memory_spread_page</I>
|
|
|
|
and
|
|
<I>cpuset.memory_spread_slab</I>.
|
|
|
|
<P>
|
|
|
|
If the per-cpuset Boolean flag file
|
|
<I>cpuset.memory_spread_page</I>
|
|
|
|
is set, then
|
|
the kernel will spread the filesystem buffers (page cache) evenly
|
|
over all the nodes that the faulting process is allowed to use, instead
|
|
of preferring to put those pages on the node where the process is running.
|
|
<P>
|
|
|
|
If the per-cpuset Boolean flag file
|
|
<I>cpuset.memory_spread_slab</I>
|
|
|
|
is set,
|
|
then the kernel will spread some filesystem-related slab caches,
|
|
such as those for inodes and directory entries, evenly over all the nodes
|
|
that the faulting process is allowed to use, instead of preferring to
|
|
put those pages on the node where the process is running.
|
|
<P>
|
|
|
|
The setting of these flags does not affect the data segment
|
|
(see
|
|
<B><A HREF="/cgi-bin/man/man2html?2+brk">brk</A></B>(2))
|
|
|
|
or stack segment pages of a process.
|
|
<P>
|
|
|
|
By default, both kinds of memory spreading are off and the kernel
|
|
prefers to allocate memory pages on the node local to where the
|
|
requesting process is running.
|
|
If that node is not allowed by the
|
|
process's NUMA memory policy or cpuset configuration or if there are
|
|
insufficient free memory pages on that node, then the kernel looks
|
|
for the nearest node that is allowed and has sufficient free memory.
|
|
<P>
|
|
|
|
When new cpusets are created, they inherit the memory spread settings
|
|
of their parent.
|
|
<P>
|
|
|
|
Setting memory spreading causes allocations for the affected page or
|
|
slab caches to ignore the process's NUMA memory policy and be spread
|
|
instead.
|
|
However, the effect of these changes in memory placement
|
|
caused by cpuset-specified memory spreading is hidden from the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mbind">mbind</A></B>(2)
|
|
|
|
or
|
|
<B><A HREF="/cgi-bin/man/man2html?2+set_mempolicy">set_mempolicy</A></B>(2)
|
|
|
|
calls.
|
|
These two NUMA memory policy calls always appear to behave as if
|
|
no cpuset-specified memory spreading is in effect, even if it is.
|
|
If cpuset memory spreading is subsequently turned off, the NUMA
|
|
memory policy most recently specified by these calls is automatically
|
|
reapplied.
|
|
<P>
|
|
|
|
Both
|
|
<I>cpuset.memory_spread_page</I>
|
|
|
|
and
|
|
<I>cpuset.memory_spread_slab</I>
|
|
|
|
are Boolean flag files.
|
|
By default, they contain "0", meaning that the feature is off
|
|
for that cpuset.
|
|
If a "1" is written to that file, that turns the named feature on.
|
|
<P>
|
|
|
|
Cpuset-specified memory spreading behaves similarly to what is known
|
|
(in other contexts) as round-robin or interleave memory placement.
|
|
<P>
|
|
|
|
Cpuset-specified memory spreading can provide substantial performance
|
|
improvements for jobs that:
|
|
<DL COMPACT>
|
|
<DT id="22">a)<DD>
|
|
need to place thread-local data on
|
|
memory nodes close to the CPUs which are running the threads that most
|
|
frequently access that data; but also
|
|
<DT id="23">b)<DD>
|
|
need to access large filesystem data sets that must to be spread
|
|
across the several nodes in the job's cpuset in order to fit.
|
|
</DL>
|
|
<P>
|
|
|
|
Without this policy,
|
|
the memory allocation across the nodes in the job's cpuset
|
|
can become very uneven,
|
|
especially for jobs that might have just a single
|
|
thread initializing or reading in the data set.
|
|
|
|
<A NAME="lbAK"> </A>
|
|
<H3>Memory migration</H3>
|
|
|
|
Normally, under the default setting (disabled) of
|
|
<I>cpuset.memory_migrate</I>,
|
|
|
|
once a page is allocated (given a physical page
|
|
of main memory), then that page stays on whatever node it
|
|
was allocated, so long as it remains allocated, even if the
|
|
cpuset's memory-placement policy
|
|
<I>mems</I>
|
|
|
|
subsequently changes.
|
|
<P>
|
|
|
|
When memory migration is enabled in a cpuset, if the
|
|
<I>mems</I>
|
|
|
|
setting of the cpuset is changed, then any memory page in use by any
|
|
process in the cpuset that is on a memory node that is no longer
|
|
allowed will be migrated to a memory node that is allowed.
|
|
<P>
|
|
|
|
Furthermore, if a process is moved into a cpuset with
|
|
<I>memory_migrate</I>
|
|
|
|
enabled, any memory pages it uses that were on memory nodes allowed
|
|
in its previous cpuset, but which are not allowed in its new cpuset,
|
|
will be migrated to a memory node allowed in the new cpuset.
|
|
<P>
|
|
|
|
The relative placement of a migrated page within
|
|
the cpuset is preserved during these migration operations if possible.
|
|
For example,
|
|
if the page was on the second valid node of the prior cpuset,
|
|
then the page will be placed on the second valid node of the new cpuset,
|
|
if possible.
|
|
|
|
<A NAME="lbAL"> </A>
|
|
<H3>Scheduler load balancing</H3>
|
|
|
|
The kernel scheduler automatically load balances processes.
|
|
If one CPU is underutilized,
|
|
the kernel will look for processes on other more
|
|
overloaded CPUs and move those processes to the underutilized CPU,
|
|
within the constraints of such placement mechanisms as cpusets and
|
|
<B><A HREF="/cgi-bin/man/man2html?2+sched_setaffinity">sched_setaffinity</A></B>(2).
|
|
|
|
<P>
|
|
|
|
The algorithmic cost of load balancing and its impact on key shared
|
|
kernel data structures such as the process list increases more than
|
|
linearly with the number of CPUs being balanced.
|
|
For example, it
|
|
costs more to load balance across one large set of CPUs than it does
|
|
to balance across two smaller sets of CPUs, each of half the size
|
|
of the larger set.
|
|
(The precise relationship between the number of CPUs being balanced
|
|
and the cost of load balancing depends
|
|
on implementation details of the kernel process scheduler, which is
|
|
subject to change over time, as improved kernel scheduler algorithms
|
|
are implemented.)
|
|
<P>
|
|
|
|
The per-cpuset flag
|
|
<I>sched_load_balance</I>
|
|
|
|
provides a mechanism to suppress this automatic scheduler load
|
|
balancing in cases where it is not needed and suppressing it would have
|
|
worthwhile performance benefits.
|
|
<P>
|
|
|
|
By default, load balancing is done across all CPUs, except those
|
|
marked isolated using the kernel boot time "isolcpus=" argument.
|
|
(See <B>Scheduler Relax Domain Level</B>, below, to change this default.)
|
|
<P>
|
|
|
|
This default load balancing across all CPUs is not well suited to
|
|
the following two situations:
|
|
<DL COMPACT>
|
|
<DT id="24">*<DD>
|
|
On large systems, load balancing across many CPUs is expensive.
|
|
If the system is managed using cpusets to place independent jobs
|
|
on separate sets of CPUs, full load balancing is unnecessary.
|
|
<DT id="25">*<DD>
|
|
Systems supporting real-time on some CPUs need to minimize
|
|
system overhead on those CPUs, including avoiding process load
|
|
balancing if that is not needed.
|
|
</DL>
|
|
<P>
|
|
|
|
When the per-cpuset flag
|
|
<I>sched_load_balance</I>
|
|
|
|
is enabled (the default setting),
|
|
it requests load balancing across
|
|
all the CPUs in that cpuset's allowed CPUs,
|
|
ensuring that load balancing can move a process (not otherwise pinned,
|
|
as by
|
|
<B><A HREF="/cgi-bin/man/man2html?2+sched_setaffinity">sched_setaffinity</A></B>(2))
|
|
|
|
from any CPU in that cpuset to any other.
|
|
<P>
|
|
|
|
When the per-cpuset flag
|
|
<I>sched_load_balance</I>
|
|
|
|
is disabled, then the
|
|
scheduler will avoid load balancing across the CPUs in that cpuset,
|
|
<I>except</I> in so far as is necessary because some overlapping cpuset
|
|
has
|
|
<I>sched_load_balance</I>
|
|
|
|
enabled.
|
|
<P>
|
|
|
|
So, for example, if the top cpuset has the flag
|
|
<I>sched_load_balance</I>
|
|
|
|
enabled, then the scheduler will load balance across all
|
|
CPUs, and the setting of the
|
|
<I>sched_load_balance</I>
|
|
|
|
flag in other cpusets has no effect,
|
|
as we're already fully load balancing.
|
|
<P>
|
|
|
|
Therefore in the above two situations, the flag
|
|
<I>sched_load_balance</I>
|
|
|
|
should be disabled in the top cpuset, and only some of the smaller,
|
|
child cpusets would have this flag enabled.
|
|
<P>
|
|
|
|
When doing this, you don't usually want to leave any unpinned processes in
|
|
the top cpuset that might use nontrivial amounts of CPU, as such processes
|
|
may be artificially constrained to some subset of CPUs, depending on
|
|
the particulars of this flag setting in descendant cpusets.
|
|
Even if such a process could use spare CPU cycles in some other CPUs,
|
|
the kernel scheduler might not consider the possibility of
|
|
load balancing that process to the underused CPU.
|
|
<P>
|
|
|
|
Of course, processes pinned to a particular CPU can be left in a cpuset
|
|
that disables
|
|
<I>sched_load_balance</I>
|
|
|
|
as those processes aren't going anywhere else anyway.
|
|
|
|
<A NAME="lbAM"> </A>
|
|
<H3>Scheduler relax domain level</H3>
|
|
|
|
The kernel scheduler performs immediate load balancing whenever
|
|
a CPU becomes free or another task becomes runnable.
|
|
This load
|
|
balancing works to ensure that as many CPUs as possible are usefully
|
|
employed running tasks.
|
|
The kernel also performs periodic load
|
|
balancing off the software clock described in
|
|
<B><A HREF="/cgi-bin/man/man2html?7+time">time</A></B>(7).
|
|
|
|
The setting of
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
applies only to immediate load balancing.
|
|
Regardless of the
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
setting, periodic load balancing is attempted over all CPUs
|
|
(unless disabled by turning off
|
|
<I>sched_load_balance</I>.)
|
|
|
|
In any case, of course, tasks will be scheduled to run only on
|
|
CPUs allowed by their cpuset, as modified by
|
|
<B><A HREF="/cgi-bin/man/man2html?2+sched_setaffinity">sched_setaffinity</A></B>(2)
|
|
|
|
system calls.
|
|
<P>
|
|
|
|
On small systems, such as those with just a few CPUs, immediate load
|
|
balancing is useful to improve system interactivity and to minimize
|
|
wasteful idle CPU cycles.
|
|
But on large systems, attempting immediate
|
|
load balancing across a large number of CPUs can be more costly than
|
|
it is worth, depending on the particular performance characteristics
|
|
of the job mix and the hardware.
|
|
<P>
|
|
|
|
The exact meaning of the small integer values of
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
will depend on internal
|
|
implementation details of the kernel scheduler code and on the
|
|
non-uniform architecture of the hardware.
|
|
Both of these will evolve
|
|
over time and vary by system architecture and kernel version.
|
|
<P>
|
|
|
|
As of this writing, when this capability was introduced in Linux
|
|
2.6.26, on certain popular architectures, the positive values of
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
have the following meanings.
|
|
<P>
|
|
|
|
|
|
<DL COMPACT>
|
|
<DT id="26"><B>(1)</B><DD>
|
|
Perform immediate load balancing across Hyper-Thread
|
|
siblings on the same core.
|
|
<DT id="27"><B>(2)</B><DD>
|
|
Perform immediate load balancing across other cores in the same package.
|
|
<DT id="28"><B>(3)</B><DD>
|
|
Perform immediate load balancing across other CPUs
|
|
on the same node or blade.
|
|
<DT id="29"><B>(4)</B><DD>
|
|
Perform immediate load balancing across over several
|
|
(implementation detail) nodes [On NUMA systems].
|
|
<DT id="30"><B>(5)</B><DD>
|
|
Perform immediate load balancing across over all CPUs
|
|
in system [On NUMA systems].
|
|
|
|
</DL>
|
|
<P>
|
|
|
|
The
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
value of zero (0) always means
|
|
don't perform immediate load balancing,
|
|
hence that load balancing is done only periodically,
|
|
not immediately when a CPU becomes available or another task becomes
|
|
runnable.
|
|
<P>
|
|
|
|
The
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
value of minus one (-1)
|
|
always means use the system default value.
|
|
The system default value can vary by architecture and kernel version.
|
|
This system default value can be changed by kernel
|
|
boot-time "relax_domain_level=" argument.
|
|
<P>
|
|
|
|
In the case of multiple overlapping cpusets which have conflicting
|
|
<I>sched_relax_domain_level</I>
|
|
|
|
values, then the highest such value
|
|
applies to all CPUs in any of the overlapping cpusets.
|
|
In such cases,
|
|
the value <B>minus one (-1)</B> is the lowest value, overridden by any
|
|
other value, and the value <B>zero (0)</B> is the next lowest value.
|
|
<A NAME="lbAN"> </A>
|
|
<H2>FORMATS</H2>
|
|
|
|
The following formats are used to represent sets of
|
|
CPUs and memory nodes.
|
|
|
|
<A NAME="lbAO"> </A>
|
|
<H3>Mask format</H3>
|
|
|
|
The <B>Mask Format</B> is used to represent CPU and memory-node bit masks
|
|
in the
|
|
<I>/proc/<pid>/status</I>
|
|
|
|
file.
|
|
<P>
|
|
|
|
This format displays each 32-bit
|
|
word in hexadecimal (using ASCII characters "0" - "9" and "a" - "f");
|
|
words are filled with leading zeros, if required.
|
|
For masks longer than one word, a comma separator is used between words.
|
|
Words are displayed in big-endian
|
|
order, which has the most significant bit first.
|
|
The hex digits within a word are also in big-endian order.
|
|
<P>
|
|
|
|
The number of 32-bit words displayed is the minimum number needed to
|
|
display all bits of the bit mask, based on the size of the bit mask.
|
|
<P>
|
|
|
|
Examples of the <B>Mask Format</B>:
|
|
<P>
|
|
|
|
|
|
|
|
00000001 # just bit 0 set
|
|
40000000,00000000,00000000 # just bit 94 set
|
|
00000001,00000000,00000000 # just bit 64 set
|
|
000000ff,00000000 # bits 32-39 set
|
|
00000000,000e3862 # 1,5,6,11-13,17-19 set
|
|
|
|
|
|
<P>
|
|
|
|
A mask with bits 0, 1, 2, 4, 8, 16, 32, and 64 set displays as:
|
|
<P>
|
|
|
|
|
|
|
|
00000001,00000001,00010117
|
|
|
|
|
|
<P>
|
|
|
|
The first "1" is for bit 64, the
|
|
second for bit 32, the third for bit 16, the fourth for bit 8, the
|
|
fifth for bit 4, and the "7" is for bits 2, 1, and 0.
|
|
|
|
<A NAME="lbAP"> </A>
|
|
<H3>List format</H3>
|
|
|
|
The <B>List Format</B> for
|
|
<I>cpus</I>
|
|
|
|
and
|
|
<I>mems</I>
|
|
|
|
is a comma-separated list of CPU or memory-node
|
|
numbers and ranges of numbers, in ASCII decimal.
|
|
<P>
|
|
|
|
Examples of the <B>List Format</B>:
|
|
<P>
|
|
|
|
|
|
|
|
0-4,9 # bits 0, 1, 2, 3, 4, and 9 set
|
|
0-2,7,12-14 # bits 0, 1, 2, 7, 12, 13, and 14 set
|
|
|
|
|
|
|
|
<A NAME="lbAQ"> </A>
|
|
<H2>RULES</H2>
|
|
|
|
The following rules apply to each cpuset:
|
|
<DL COMPACT>
|
|
<DT id="31">*<DD>
|
|
Its CPUs and memory nodes must be a (possibly equal)
|
|
subset of its parent's.
|
|
<DT id="32">*<DD>
|
|
It can be marked
|
|
<I>cpu_exclusive</I>
|
|
|
|
only if its parent is.
|
|
<DT id="33">*<DD>
|
|
It can be marked
|
|
<I>mem_exclusive</I>
|
|
|
|
only if its parent is.
|
|
<DT id="34">*<DD>
|
|
If it is
|
|
<I>cpu_exclusive</I>,
|
|
|
|
its CPUs may not overlap any sibling.
|
|
<DT id="35">*<DD>
|
|
If it is
|
|
<I>memory_exclusive</I>,
|
|
|
|
its memory nodes may not overlap any sibling.
|
|
|
|
</DL>
|
|
<A NAME="lbAR"> </A>
|
|
<H2>PERMISSIONS</H2>
|
|
|
|
The permissions of a cpuset are determined by the permissions
|
|
of the directories and pseudo-files in the cpuset filesystem,
|
|
normally mounted at
|
|
<I>/dev/cpuset</I>.
|
|
|
|
<P>
|
|
|
|
For instance, a process can put itself in some other cpuset (than
|
|
its current one) if it can write the
|
|
<I>tasks</I>
|
|
|
|
file for that cpuset.
|
|
This requires execute permission on the encompassing directories
|
|
and write permission on the
|
|
<I>tasks</I>
|
|
|
|
file.
|
|
<P>
|
|
|
|
An additional constraint is applied to requests to place some
|
|
other process in a cpuset.
|
|
One process may not attach another to
|
|
a cpuset unless it would have permission to send that process
|
|
a signal (see
|
|
<B><A HREF="/cgi-bin/man/man2html?2+kill">kill</A></B>(2)).
|
|
|
|
<P>
|
|
|
|
A process may create a child cpuset if it can access and write the
|
|
parent cpuset directory.
|
|
It can modify the CPUs or memory nodes
|
|
in a cpuset if it can access that cpuset's directory (execute
|
|
permissions on the each of the parent directories) and write the
|
|
corresponding
|
|
<I>cpus</I>
|
|
|
|
or
|
|
<I>mems</I>
|
|
|
|
file.
|
|
<P>
|
|
|
|
There is one minor difference between the manner in which these
|
|
permissions are evaluated and the manner in which normal filesystem
|
|
operation permissions are evaluated.
|
|
The kernel interprets
|
|
relative pathnames starting at a process's current working directory.
|
|
Even if one is operating on a cpuset file, relative pathnames
|
|
are interpreted relative to the process's current working directory,
|
|
not relative to the process's current cpuset.
|
|
The only ways that
|
|
cpuset paths relative to a process's current cpuset can be used are
|
|
if either the process's current working directory is its cpuset
|
|
(it first did a
|
|
<B>cd</B>
|
|
|
|
or
|
|
<B><A HREF="/cgi-bin/man/man2html?2+chdir">chdir</A></B>(2)
|
|
|
|
to its cpuset directory beneath
|
|
<I>/dev/cpuset</I>,
|
|
|
|
which is a bit unusual)
|
|
or if some user code converts the relative cpuset path to a
|
|
full filesystem path.
|
|
<P>
|
|
|
|
In theory, this means that user code should specify cpusets
|
|
using absolute pathnames, which requires knowing the mount point of
|
|
the cpuset filesystem (usually, but not necessarily,
|
|
<I>/dev/cpuset</I>).
|
|
|
|
In practice, all user level code that this author is aware of
|
|
simply assumes that if the cpuset filesystem is mounted, then
|
|
it is mounted at
|
|
<I>/dev/cpuset</I>.
|
|
|
|
Furthermore, it is common practice for carefully written
|
|
user code to verify the presence of the pseudo-file
|
|
<I>/dev/cpuset/tasks</I>
|
|
|
|
in order to verify that the cpuset pseudo-filesystem
|
|
is currently mounted.
|
|
|
|
<A NAME="lbAS"> </A>
|
|
<H2>WARNINGS</H2>
|
|
|
|
<A NAME="lbAT"> </A>
|
|
<H3>Enabling memory_pressure</H3>
|
|
|
|
By default, the per-cpuset file
|
|
<I>cpuset.memory_pressure</I>
|
|
|
|
always contains zero (0).
|
|
Unless this feature is enabled by writing "1" to the pseudo-file
|
|
<I>/dev/cpuset/cpuset.memory_pressure_enabled</I>,
|
|
|
|
the kernel does
|
|
not compute per-cpuset
|
|
<I>memory_pressure</I>.
|
|
|
|
<A NAME="lbAU"> </A>
|
|
<H3>Using the echo command</H3>
|
|
|
|
When using the
|
|
<B>echo</B>
|
|
|
|
command at the shell prompt to change the values of cpuset files,
|
|
beware that the built-in
|
|
<B>echo</B>
|
|
|
|
command in some shells does not display an error message if the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
system call fails.
|
|
|
|
For example, if the command:
|
|
<P>
|
|
|
|
|
|
|
|
echo 19 > cpuset.mems
|
|
|
|
|
|
<P>
|
|
|
|
failed because memory node 19 was not allowed (perhaps
|
|
the current system does not have a memory node 19), then the
|
|
<B>echo</B>
|
|
|
|
command might not display any error.
|
|
It is better to use the
|
|
<B>/bin/echo</B>
|
|
|
|
external command to change cpuset file settings, as this
|
|
command will display
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
errors, as in the example:
|
|
<P>
|
|
|
|
|
|
|
|
/bin/echo 19 > cpuset.mems
|
|
/bin/echo: write error: Invalid argument
|
|
|
|
|
|
|
|
<A NAME="lbAV"> </A>
|
|
<H2>EXCEPTIONS</H2>
|
|
|
|
<A NAME="lbAW"> </A>
|
|
<H3>Memory placement</H3>
|
|
|
|
Not all allocations of system memory are constrained by cpusets,
|
|
for the following reasons.
|
|
<P>
|
|
|
|
If hot-plug functionality is used to remove all the CPUs that are
|
|
currently assigned to a cpuset, then the kernel will automatically
|
|
update the
|
|
<I>cpus_allowed</I>
|
|
|
|
of all processes attached to CPUs in that cpuset
|
|
to allow all CPUs.
|
|
When memory hot-plug functionality for removing
|
|
memory nodes is available, a similar exception is expected to apply
|
|
there as well.
|
|
In general, the kernel prefers to violate cpuset placement,
|
|
rather than starving a process that has had all its allowed CPUs or
|
|
memory nodes taken offline.
|
|
User code should reconfigure cpusets to refer only to online CPUs
|
|
and memory nodes when using hot-plug to add or remove such resources.
|
|
<P>
|
|
|
|
A few kernel-critical, internal memory-allocation requests, marked
|
|
GFP_ATOMIC, must be satisfied immediately.
|
|
The kernel may drop some
|
|
request or malfunction if one of these allocations fail.
|
|
If such a request cannot be satisfied within the current process's cpuset,
|
|
then we relax the cpuset, and look for memory anywhere we can find it.
|
|
It's better to violate the cpuset than stress the kernel.
|
|
<P>
|
|
|
|
Allocations of memory requested by kernel drivers while processing
|
|
an interrupt lack any relevant process context, and are not confined
|
|
by cpusets.
|
|
<A NAME="lbAX"> </A>
|
|
<H3>Renaming cpusets</H3>
|
|
|
|
You can use the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+rename">rename</A></B>(2)
|
|
|
|
system call to rename cpusets.
|
|
Only simple renaming is supported; that is, changing the name of a cpuset
|
|
directory is permitted, but moving a directory into
|
|
a different directory is not permitted.
|
|
|
|
<A NAME="lbAY"> </A>
|
|
<H2>ERRORS</H2>
|
|
|
|
The Linux kernel implementation of cpusets sets
|
|
<I>errno</I>
|
|
|
|
to specify the reason for a failed system call affecting cpusets.
|
|
<P>
|
|
|
|
The possible
|
|
<I>errno</I>
|
|
|
|
settings and their meaning when set on
|
|
a failed cpuset call are as listed below.
|
|
<DL COMPACT>
|
|
<DT id="36"><B>E2BIG</B>
|
|
|
|
<DD>
|
|
Attempted a
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
on a special cpuset file
|
|
with a length larger than some kernel-determined upper
|
|
limit on the length of such writes.
|
|
<DT id="37"><B>EACCES</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
the process ID (PID) of a process to a cpuset
|
|
<I>tasks</I>
|
|
|
|
file when one lacks permission to move that process.
|
|
<DT id="38"><B>EACCES</B>
|
|
|
|
<DD>
|
|
Attempted to add, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2),
|
|
|
|
a CPU or memory node to a cpuset, when that CPU or memory node was
|
|
not already in its parent.
|
|
<DT id="39"><B>EACCES</B>
|
|
|
|
<DD>
|
|
Attempted to set, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2),
|
|
|
|
<I>cpuset.cpu_exclusive</I>
|
|
|
|
or
|
|
<I>cpuset.mem_exclusive</I>
|
|
|
|
on a cpuset whose parent lacks the same setting.
|
|
<DT id="40"><B>EACCES</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a
|
|
<I>cpuset.memory_pressure</I>
|
|
|
|
file.
|
|
<DT id="41"><B>EACCES</B>
|
|
|
|
<DD>
|
|
Attempted to create a file in a cpuset directory.
|
|
<DT id="42"><B>EBUSY</B>
|
|
|
|
<DD>
|
|
Attempted to remove, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+rmdir">rmdir</A></B>(2),
|
|
|
|
a cpuset with attached processes.
|
|
<DT id="43"><B>EBUSY</B>
|
|
|
|
<DD>
|
|
Attempted to remove, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+rmdir">rmdir</A></B>(2),
|
|
|
|
a cpuset with child cpusets.
|
|
<DT id="44"><B>EBUSY</B>
|
|
|
|
<DD>
|
|
Attempted to remove
|
|
a CPU or memory node from a cpuset
|
|
that is also in a child of that cpuset.
|
|
<DT id="45"><B>EEXIST</B>
|
|
|
|
<DD>
|
|
Attempted to create, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mkdir">mkdir</A></B>(2),
|
|
|
|
a cpuset that already exists.
|
|
<DT id="46"><B>EEXIST</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+rename">rename</A></B>(2)
|
|
|
|
a cpuset to a name that already exists.
|
|
<DT id="47"><B>EFAULT</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+read">read</A></B>(2)
|
|
|
|
or
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a cpuset file using
|
|
a buffer that is outside the writing processes accessible address space.
|
|
<DT id="48"><B>EINVAL</B>
|
|
|
|
<DD>
|
|
Attempted to change a cpuset, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2),
|
|
|
|
in a way that would violate a
|
|
<I>cpu_exclusive</I>
|
|
|
|
or
|
|
<I>mem_exclusive</I>
|
|
|
|
attribute of that cpuset or any of its siblings.
|
|
<DT id="49"><B>EINVAL</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
an empty
|
|
<I>cpuset.cpus</I>
|
|
|
|
or
|
|
<I>cpuset.mems</I>
|
|
|
|
list to a cpuset which has attached processes or child cpusets.
|
|
<DT id="50"><B>EINVAL</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a
|
|
<I>cpuset.cpus</I>
|
|
|
|
or
|
|
<I>cpuset.mems</I>
|
|
|
|
list which included a range with the second number smaller than
|
|
the first number.
|
|
<DT id="51"><B>EINVAL</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a
|
|
<I>cpuset.cpus</I>
|
|
|
|
or
|
|
<I>cpuset.mems</I>
|
|
|
|
list which included an invalid character in the string.
|
|
<DT id="52"><B>EINVAL</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a list to a
|
|
<I>cpuset.cpus</I>
|
|
|
|
file that did not include any online CPUs.
|
|
<DT id="53"><B>EINVAL</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a list to a
|
|
<I>cpuset.mems</I>
|
|
|
|
file that did not include any online memory nodes.
|
|
<DT id="54"><B>EINVAL</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a list to a
|
|
<I>cpuset.mems</I>
|
|
|
|
file that included a node that held no memory.
|
|
<DT id="55"><B>EIO</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
a string to a cpuset
|
|
<I>tasks</I>
|
|
|
|
file that
|
|
does not begin with an ASCII decimal integer.
|
|
<DT id="56"><B>EIO</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+rename">rename</A></B>(2)
|
|
|
|
a cpuset into a different directory.
|
|
<DT id="57"><B>ENAMETOOLONG</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+read">read</A></B>(2)
|
|
|
|
a
|
|
<I>/proc/<pid>/cpuset</I>
|
|
|
|
file for a cpuset path that is longer than the kernel page size.
|
|
<DT id="58"><B>ENAMETOOLONG</B>
|
|
|
|
<DD>
|
|
Attempted to create, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mkdir">mkdir</A></B>(2),
|
|
|
|
a cpuset whose base directory name is longer than 255 characters.
|
|
<DT id="59"><B>ENAMETOOLONG</B>
|
|
|
|
<DD>
|
|
Attempted to create, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mkdir">mkdir</A></B>(2),
|
|
|
|
a cpuset whose full pathname,
|
|
including the mount point (typically "/dev/cpuset/") prefix,
|
|
is longer than 4095 characters.
|
|
<DT id="60"><B>ENODEV</B>
|
|
|
|
<DD>
|
|
The cpuset was removed by another process at the same time as a
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
was attempted on one of the pseudo-files in the cpuset directory.
|
|
<DT id="61"><B>ENOENT</B>
|
|
|
|
<DD>
|
|
Attempted to create, using
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mkdir">mkdir</A></B>(2),
|
|
|
|
a cpuset in a parent cpuset that doesn't exist.
|
|
<DT id="62"><B>ENOENT</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+access">access</A></B>(2)
|
|
|
|
or
|
|
<B><A HREF="/cgi-bin/man/man2html?2+open">open</A></B>(2)
|
|
|
|
a nonexistent file in a cpuset directory.
|
|
<DT id="63"><B>ENOMEM</B>
|
|
|
|
<DD>
|
|
Insufficient memory is available within the kernel; can occur
|
|
on a variety of system calls affecting cpusets, but only if the
|
|
system is extremely short of memory.
|
|
<DT id="64"><B>ENOSPC</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
the process ID (PID)
|
|
of a process to a cpuset
|
|
<I>tasks</I>
|
|
|
|
file when the cpuset had an empty
|
|
<I>cpuset.cpus</I>
|
|
|
|
or empty
|
|
<I>cpuset.mems</I>
|
|
|
|
setting.
|
|
<DT id="65"><B>ENOSPC</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
an empty
|
|
<I>cpuset.cpus</I>
|
|
|
|
or
|
|
<I>cpuset.mems</I>
|
|
|
|
setting to a cpuset that
|
|
has tasks attached.
|
|
<DT id="66"><B>ENOTDIR</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+rename">rename</A></B>(2)
|
|
|
|
a nonexistent cpuset.
|
|
<DT id="67"><B>EPERM</B>
|
|
|
|
<DD>
|
|
Attempted to remove a file from a cpuset directory.
|
|
<DT id="68"><B>ERANGE</B>
|
|
|
|
<DD>
|
|
Specified a
|
|
<I>cpuset.cpus</I>
|
|
|
|
or
|
|
<I>cpuset.mems</I>
|
|
|
|
list to the kernel which included a number too large for the kernel
|
|
to set in its bit masks.
|
|
<DT id="69"><B>ESRCH</B>
|
|
|
|
<DD>
|
|
Attempted to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
the process ID (PID) of a nonexistent process to a cpuset
|
|
<I>tasks</I>
|
|
|
|
file.
|
|
|
|
</DL>
|
|
<A NAME="lbAZ"> </A>
|
|
<H2>VERSIONS</H2>
|
|
|
|
Cpusets appeared in version 2.6.12 of the Linux kernel.
|
|
|
|
<A NAME="lbBA"> </A>
|
|
<H2>NOTES</H2>
|
|
|
|
Despite its name, the
|
|
<I>pid</I>
|
|
|
|
parameter is actually a thread ID,
|
|
and each thread in a threaded group can be attached to a different
|
|
cpuset.
|
|
The value returned from a call to
|
|
<B><A HREF="/cgi-bin/man/man2html?2+gettid">gettid</A></B>(2)
|
|
|
|
can be passed in the argument
|
|
<I>pid</I>.
|
|
|
|
|
|
<A NAME="lbBB"> </A>
|
|
<H2>BUGS</H2>
|
|
|
|
<I>cpuset.memory_pressure</I>
|
|
|
|
cpuset files can be opened
|
|
for writing, creation, or truncation, but then the
|
|
<B><A HREF="/cgi-bin/man/man2html?2+write">write</A></B>(2)
|
|
|
|
fails with
|
|
<I>errno</I>
|
|
|
|
set to
|
|
<B>EACCES</B>,
|
|
|
|
and the creation and truncation options on
|
|
<B><A HREF="/cgi-bin/man/man2html?2+open">open</A></B>(2)
|
|
|
|
have no effect.
|
|
|
|
<A NAME="lbBC"> </A>
|
|
<H2>EXAMPLE</H2>
|
|
|
|
The following examples demonstrate querying and setting cpuset
|
|
options using shell commands.
|
|
<A NAME="lbBD"> </A>
|
|
<H3>Creating and attaching to a cpuset.</H3>
|
|
|
|
To create a new cpuset and attach the current command shell to it,
|
|
the steps are:
|
|
<P>
|
|
|
|
|
|
<DL COMPACT>
|
|
<DT id="70">1)<DD>
|
|
mkdir /dev/cpuset (if not already done)
|
|
<DT id="71">2)<DD>
|
|
mount -t cpuset none /dev/cpuset (if not already done)
|
|
<DT id="72">3)<DD>
|
|
Create the new cpuset using
|
|
<B><A HREF="/cgi-bin/man/man2html?1+mkdir">mkdir</A></B>(1).
|
|
|
|
<DT id="73">4)<DD>
|
|
Assign CPUs and memory nodes to the new cpuset.
|
|
<DT id="74">5)<DD>
|
|
Attach the shell to the new cpuset.
|
|
|
|
</DL>
|
|
<P>
|
|
|
|
For example, the following sequence of commands will set up a cpuset
|
|
named "Charlie", containing just CPUs 2 and 3, and memory node 1,
|
|
and then attach the current shell to that cpuset.
|
|
<P>
|
|
|
|
|
|
|
|
$<B> mkdir /dev/cpuset</B>
|
|
|
|
$<B> mount -t cpuset cpuset /dev/cpuset</B>
|
|
|
|
$<B> cd /dev/cpuset</B>
|
|
|
|
$<B> mkdir Charlie</B>
|
|
|
|
$<B> cd Charlie</B>
|
|
|
|
$<B> /bin/echo 2-3 > cpuset.cpus</B>
|
|
|
|
$<B> /bin/echo 1 > cpuset.mems</B>
|
|
|
|
$<B> /bin/echo $$ > tasks</B>
|
|
|
|
# The current shell is now running in cpuset Charlie
|
|
# The next line should display '/Charlie'
|
|
$<B> cat /proc/self/cpuset</B>
|
|
|
|
|
|
|
|
|
|
<A NAME="lbBE"> </A>
|
|
<H3>Migrating a job to different memory nodes.</H3>
|
|
|
|
To migrate a job (the set of processes attached to a cpuset)
|
|
to different CPUs and memory nodes in the system, including moving
|
|
the memory pages currently allocated to that job,
|
|
perform the following steps.
|
|
<P>
|
|
|
|
|
|
<DL COMPACT>
|
|
<DT id="75">1)<DD>
|
|
Let's say we want to move the job in cpuset
|
|
<I>alpha</I>
|
|
|
|
(CPUs 4-7 and memory nodes 2-3) to a new cpuset
|
|
<I>beta</I>
|
|
|
|
(CPUs 16-19 and memory nodes 8-9).
|
|
<DT id="76">2)<DD>
|
|
First create the new cpuset
|
|
<I>beta</I>.
|
|
|
|
<DT id="77">3)<DD>
|
|
Then allow CPUs 16-19 and memory nodes 8-9 in
|
|
<I>beta</I>.
|
|
|
|
<DT id="78">4)<DD>
|
|
Then enable
|
|
<I>memory_migration</I>
|
|
|
|
in
|
|
<I>beta</I>.
|
|
|
|
<DT id="79">5)<DD>
|
|
Then move each process from
|
|
<I>alpha</I>
|
|
|
|
to
|
|
<I>beta</I>.
|
|
|
|
|
|
</DL>
|
|
<P>
|
|
|
|
The following sequence of commands accomplishes this.
|
|
<P>
|
|
|
|
|
|
|
|
$<B> cd /dev/cpuset</B>
|
|
|
|
$<B> mkdir beta</B>
|
|
|
|
$<B> cd beta</B>
|
|
|
|
$<B> /bin/echo 16-19 > cpuset.cpus</B>
|
|
|
|
$<B> /bin/echo 8-9 > cpuset.mems</B>
|
|
|
|
$<B> /bin/echo 1 > cpuset.memory_migrate</B>
|
|
|
|
$<B> while read i; do /bin/echo $i; done < ../alpha/tasks > tasks</B>
|
|
|
|
|
|
|
|
<P>
|
|
|
|
The above should move any processes in
|
|
<I>alpha</I>
|
|
|
|
to
|
|
<I>beta</I>,
|
|
|
|
and any memory held by these processes on memory nodes 2-3 to memory
|
|
nodes 8-9, respectively.
|
|
<P>
|
|
|
|
Notice that the last step of the above sequence did not do:
|
|
<P>
|
|
|
|
|
|
|
|
$<B> cp ../alpha/tasks tasks</B>
|
|
|
|
|
|
|
|
<P>
|
|
|
|
The
|
|
<I>while</I>
|
|
|
|
loop, rather than the seemingly easier use of the
|
|
<B><A HREF="/cgi-bin/man/man2html?1+cp">cp</A></B>(1)
|
|
|
|
command, was necessary because
|
|
only one process PID at a time may be written to the
|
|
<I>tasks</I>
|
|
|
|
file.
|
|
<P>
|
|
|
|
The same effect (writing one PID at a time) as the
|
|
<I>while</I>
|
|
|
|
loop can be accomplished more efficiently, in fewer keystrokes and in
|
|
syntax that works on any shell, but alas more obscurely, by using the
|
|
<B>-u</B>
|
|
|
|
(unbuffered) option of
|
|
<B><A HREF="/cgi-bin/man/man2html?1+sed">sed</A></B>(1):
|
|
|
|
<P>
|
|
|
|
|
|
|
|
$<B> sed -un p < ../alpha/tasks > tasks</B>
|
|
|
|
|
|
|
|
|
|
<A NAME="lbBF"> </A>
|
|
<H2>SEE ALSO</H2>
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?1+taskset">taskset</A></B>(1),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+get_mempolicy">get_mempolicy</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+getcpu">getcpu</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+mbind">mbind</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+sched_getaffinity">sched_getaffinity</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+sched_setaffinity">sched_setaffinity</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+sched_setscheduler">sched_setscheduler</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?2+set_mempolicy">set_mempolicy</A></B>(2),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?3+CPU_SET">CPU_SET</A></B>(3),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?5+proc">proc</A></B>(5),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?7+cgroups">cgroups</A></B>(7),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?7+numa">numa</A></B>(7),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?7+sched">sched</A></B>(7),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?8+migratepages">migratepages</A></B>(8),
|
|
|
|
<B><A HREF="/cgi-bin/man/man2html?8+numactl">numactl</A></B>(8)
|
|
|
|
<P>
|
|
|
|
<I>Documentation/cgroup-v1/cpusets.txt</I>
|
|
|
|
in the Linux kernel source tree
|
|
|
|
(or
|
|
<I>Documentation/cpusets.txt</I>
|
|
|
|
before Linux 2.6.29)
|
|
<A NAME="lbBG"> </A>
|
|
<H2>COLOPHON</H2>
|
|
|
|
This page is part of release 5.05 of the Linux
|
|
<I>man-pages</I>
|
|
|
|
project.
|
|
A description of the project,
|
|
information about reporting bugs,
|
|
and the latest version of this page,
|
|
can be found at
|
|
<A HREF="https://www.kernel.org/doc/man-pages/.">https://www.kernel.org/doc/man-pages/.</A>
|
|
<P>
|
|
|
|
<HR>
|
|
<A NAME="index"> </A><H2>Index</H2>
|
|
<DL>
|
|
<DT id="80"><A HREF="#lbAB">NAME</A><DD>
|
|
<DT id="81"><A HREF="#lbAC">DESCRIPTION</A><DD>
|
|
<DT id="82"><A HREF="#lbAD">FILES</A><DD>
|
|
<DT id="83"><A HREF="#lbAE">EXTENDED CAPABILITIES</A><DD>
|
|
<DL>
|
|
<DT id="84"><A HREF="#lbAF">Exclusive cpusets</A><DD>
|
|
<DT id="85"><A HREF="#lbAG">Hardwall</A><DD>
|
|
<DT id="86"><A HREF="#lbAH">Notify on release</A><DD>
|
|
<DT id="87"><A HREF="#lbAI">Memory pressure</A><DD>
|
|
<DT id="88"><A HREF="#lbAJ">Memory spread</A><DD>
|
|
<DT id="89"><A HREF="#lbAK">Memory migration</A><DD>
|
|
<DT id="90"><A HREF="#lbAL">Scheduler load balancing</A><DD>
|
|
<DT id="91"><A HREF="#lbAM">Scheduler relax domain level</A><DD>
|
|
</DL>
|
|
<DT id="92"><A HREF="#lbAN">FORMATS</A><DD>
|
|
<DL>
|
|
<DT id="93"><A HREF="#lbAO">Mask format</A><DD>
|
|
<DT id="94"><A HREF="#lbAP">List format</A><DD>
|
|
</DL>
|
|
<DT id="95"><A HREF="#lbAQ">RULES</A><DD>
|
|
<DT id="96"><A HREF="#lbAR">PERMISSIONS</A><DD>
|
|
<DT id="97"><A HREF="#lbAS">WARNINGS</A><DD>
|
|
<DL>
|
|
<DT id="98"><A HREF="#lbAT">Enabling memory_pressure</A><DD>
|
|
<DT id="99"><A HREF="#lbAU">Using the echo command</A><DD>
|
|
</DL>
|
|
<DT id="100"><A HREF="#lbAV">EXCEPTIONS</A><DD>
|
|
<DL>
|
|
<DT id="101"><A HREF="#lbAW">Memory placement</A><DD>
|
|
<DT id="102"><A HREF="#lbAX">Renaming cpusets</A><DD>
|
|
</DL>
|
|
<DT id="103"><A HREF="#lbAY">ERRORS</A><DD>
|
|
<DT id="104"><A HREF="#lbAZ">VERSIONS</A><DD>
|
|
<DT id="105"><A HREF="#lbBA">NOTES</A><DD>
|
|
<DT id="106"><A HREF="#lbBB">BUGS</A><DD>
|
|
<DT id="107"><A HREF="#lbBC">EXAMPLE</A><DD>
|
|
<DL>
|
|
<DT id="108"><A HREF="#lbBD">Creating and attaching to a cpuset.</A><DD>
|
|
<DT id="109"><A HREF="#lbBE">Migrating a job to different memory nodes.</A><DD>
|
|
</DL>
|
|
<DT id="110"><A HREF="#lbBF">SEE ALSO</A><DD>
|
|
<DT id="111"><A HREF="#lbBG">COLOPHON</A><DD>
|
|
</DL>
|
|
<HR>
|
|
This document was created by
|
|
<A HREF="/cgi-bin/man/man2html">man2html</A>,
|
|
using the manual pages.<BR>
|
|
Time: 00:06:08 GMT, March 31, 2021
|
|
</BODY>
|
|
</HTML>
|