Lines Matching refs:the
5 This question can be answered from a couple of perspectives: the
6 hardware view and the Linux software view.
8 From the hardware perspective, a NUMA system is a computer platform that
11 disambiguate the hardware view of these physical components/assemblies
12 from the software abstraction thereof, we'll call the components/assemblies
15 Each of the 'cells' may be viewed as an SMP [symmetric multi-processor] subset
16 of the system--although some components necessary for a stand-alone SMP system
17 may not be populated on any given cell. The cells of the NUMA system are
23 For Linux, the NUMA platforms of interest are primarily what is known as Cache
26 is handled in hardware by the processor caches and/or the system interconnect.
29 away the cell containing the CPU or IO bus making the memory access is from the
30 cell containing the target memory. For example, access to memory by CPUs
31 attached to the same cell will experience faster access times and higher
38 application software must arrange for a large majority of the memory references
39 [cache misses] to be to "local" memory--memory on the same cell, if any--or
40 to the closest cell with memory.
42 This leads to the Linux software view of a NUMA system:
44 Linux divides the system's hardware resources into multiple software
45 abstractions called "nodes". Linux maps the nodes onto the physical cells
46 of the hardware platform, abstracting away some of the details for some
57 a given node will see the same local memory access times and bandwidth.
60 the emulation of additional nodes. For NUMA emulation, linux will carve up
61 the existing nodes--or the system memory for non-NUMA platforms--into multiple
62 nodes. Each emulated node will manage a fraction of the underlying cells'
72 an ordered "zonelist". A zonelist specifies the zones/nodes to visit when a
73 selected zone/node cannot satisfy the allocation request. This situation,
78 memory, Linux must decide whether to order the zonelists such that allocations
79 fall back to the same zone type on a different node, or to a different zone
80 type on the same node. This is an important consideration because some zones,
82 a default zonelist order based on the sizes of the various zone types relative
83 to the total memory of the node and the total memory of the system. The
84 default zonelist order may be overridden using the numa_zonelist_order kernel
88 By default, Linux will attempt to satisfy memory allocation requests from the
89 node to which the CPU that executes the request is assigned. Specifically,
90 Linux will attempt to allocate from the first node in the appropriate zonelist
91 for the node where the request originates. This is called "local allocation."
92 If the "local" node cannot satisfy the request, the kernel will examine other
93 nodes' zones in the selected zonelist looking for the first zone in the list
94 that can satisfy the request.
96 Local allocation will tend to keep subsequent access to the allocated memory
97 "local" to the underlying physical resources and off the system interconnect--
98 as long as the task on whose behalf the kernel allocated some memory does not
99 later migrate away from that memory. The Linux scheduler is aware of the
100 NUMA topology of the platform--embodied in the "scheduling domains" data
101 structures [see Documentation/scheduler/sched-domains.txt]--and the scheduler
103 the scheduler does not take a task's NUMA footprint into account directly.
110 sched_setaffinity(2). Further, one can modify the kernel's default local
114 System administrators can restrict the CPUs and nodes' memories that a non-
115 privileged user can specify in the scheduling or NUMA commands and functions
119 zones [nodes] with memory in the zonelists. This means that for a memoryless
120 node the "local memory node"--the node of the first zone in CPU's node's
121 zonelist--will not be the node itself. Rather, it will be the node that the
122 kernel selected as the nearest node with memory when it built the zonelists.
123 So, default, local allocations will succeed with the kernel supplying the
124 closest available memory. This is a consequence of the same mechanism that
129 behavior. Rather they want to be sure they get memory from the specified node
130 or get notified that the node has no free memory. This is usually the case when
133 A typical model for making such an allocation is to obtain the node id of the
134 node to which the "current CPU" is attached using one of the kernel's
136 the node id returned. When such an allocation fails, the requesting subsystem
138 example of this. Or, the subsystem may choose to disable or not to enable
142 If the architecture supports--does not hide--memoryless nodes, then CPUs
143 attached to memoryless nodes would always incur the fallback path overhead
146 architectures transparently, kernel subsystems can use the numa_mem_id()
147 or cpu_to_mem() function to locate the "local memory node" for the calling or
148 specified CPU. Again, this is the same node from which default, local page