Non-uniform memory access

Non-uniform memory access NUMA is a computer memory design used in multiprocessing, where the memory access time depends on the memory location relative to the processor Under NUMA, a processor can access its own local memory faster than non-local memory memory local to another processor or memory shared between processors The benefits of NUMA are limited to particular workloads, notably on servers where the data are often associated strongly with certain tasks or users1

NUMA architectures logically follow in scaling from symmetric multiprocessing SMP architectures They were developed commercially during the 1990s by Burroughs later Unisys, Convex Computer later Hewlett-Packard, Honeywell Information Systems Italy HISI later Groupe Bull, Silicon Graphics later Silicon Graphics International, Sequent Computer Systems later IBM, Data General later EMC, and Digital later Compaq, now HP Techniques developed by these companies later featured in a variety of Unix-like operating systems, and to an extent in Windows NT

The first commercial implementation of a NUMA-based Unix system was the Symmetrical Multi Processing XPS-100 family of servers, designed by Dan Gielan of VAST Corporation for Honeywell Information Systems Italy


  • 1 Basic concept
  • 2 Cache coherent NUMA ccNUMA
  • 3 NUMA vs cluster computing
  • 4 Software support
  • 5 See also
  • 6 References
  • 7 External links

Basic conceptedit

One possible architecture of a NUMA system The processors connect to the bus or crossbar by connections of varying thickness/number This shows that different CPUs have different access priorities to memory based on their relative location

Modern CPUs operate considerably faster than the main memory they use In the early days of computing and data processing, the CPU generally ran slower than its own memory The performance lines of processors and memory crossed in the 1960s with the advent of the first supercomputers Since then, CPUs increasingly have found themselves "starved for data" and having to stall while waiting for data to arrive from memory Many supercomputer designs of the 1980s and 1990s focused on providing high-speed memory access as opposed to faster processors, allowing the computers to work on large data sets at speeds other systems could not approach

Limiting the number of memory accesses provided the key to extracting high performance from a modern computer For commodity processors, this meant installing an ever-increasing amount of high-speed cache memory and using increasingly sophisticated algorithms to avoid cache misses But the dramatic increase in size of the operating systems and of the applications run on them has generally overwhelmed these cache-processing improvements Multi-processor systems without NUMA make the problem considerably worse Now a system can starve several processors at the same time, notably because only one processor can access the computer's memory at a time2

NUMA attempts to address this problem by providing separate memory for each processor, avoiding the performance hit when several processors attempt to address the same memory For problems involving spread data common for servers and similar applications, NUMA can improve the performance over a single shared memory by a factor of roughly the number of processors or separate memory banks3 Another approach to addressing this problem, used mainly in non-NUMA systems, is the multi-channel memory architecture, in which a linear increase in the number of memory channels increases the memory access concurrency linearly4

Of course, not all data ends up confined to a single task, which means that more than one processor may require the same data To handle these cases, NUMA systems include additional hardware or software to move data between memory banks This operation slows the processors attached to those banks, so the overall speed increase due to NUMA depends heavily on the nature of the running tasks3

AMD implemented NUMA with its Opteron processor 2003, using HyperTransport Intel announced NUMA compatibility for its x86 and Itanium servers in late 2007 with its Nehalem and Tukwila CPUs5 Both CPU families share a common chipset; the interconnection is called Intel Quick Path Interconnect QPI6

Cache coherent NUMA ccNUMAedit

Topology of a ccNUMA Bulldozer server extracting using the hwloc tool For more details on this topic, see Cache memory § Directory-based cache coherence – message-passing

Nearly all CPU architectures use a small amount of very fast non-shared memory known as cache to exploit locality of reference in memory accesses With NUMA, maintaining cache coherence across shared memory has a significant overhead Although simpler to design and build, non-cache-coherent NUMA systems become prohibitively complex to program in the standard von Neumann architecture programming model7

Typically, ccNUMA uses inter-processor communication between cache controllers to keep a consistent memory image when more than one cache stores the same memory location For this reason, ccNUMA may perform poorly when multiple processors attempt to access the same memory area in rapid succession Support for NUMA in operating systems attempts to reduce the frequency of this kind of access by allocating processors and memory in NUMA-friendly ways and by avoiding scheduling and locking algorithms that make NUMA-unfriendly accesses necessary8

Alternatively, cache coherency protocols such as the MESIF protocol attempt to reduce the communication required to maintain cache coherency Scalable Coherent Interface SCI is an IEEE standard defining a directory-based cache coherency protocol to avoid scalability limitations found in earlier multiprocessor systems For example, SCI is used as the basis for the NumaConnect technology910

As of 2011, ccNUMA systems are multiprocessor systems based on the AMD Opteron processor, which can be implemented without external logic, and the Intel Itanium processor, which requires the chipset to support NUMA Examples of ccNUMA-enabled chipsets are the SGI Shub Super hub, the Intel E8870, the HP sx2000 used in the Integrity and Superdome servers, and those found in NEC Itanium-based systems Earlier ccNUMA systems such as those from Silicon Graphics were based on MIPS processors and the DEC Alpha 21364 EV7 processor

NUMA vs cluster computingedit

One can view NUMA as a tightly coupled form of cluster computing The addition of virtual memory paging to a cluster architecture can allow the implementation of NUMA entirely in software However, the inter-node latency of software-based NUMA remains several orders of magnitude greater slower than that of hardware-based NUMA1

Software supportedit

Since NUMA largely influences memory access performance, certain software optimizations are needed to allow scheduling threads and processes close to their in-memory data

  • Microsoft Windows 7 and Windows Server 2008 R2 added support for NUMA architecture over 64 logical cores11
  • Java 7 added support for NUMA-aware memory allocator and garbage collector12
  • Version 25 of the Linux kernel already contained basic NUMA support,13 which was further improved in subsequent kernel releases Version 38 of the Linux kernel brought a new NUMA foundation that allowed development of more efficient NUMA policies in later kernel releases1415 Version 313 of the Linux kernel brought numerous policies that aim at putting a process near its memory, together with the handling of cases such as having memory pages shared between processes, or the use of transparent huge pages; new sysctl settings allow NUMA balancing to be enabled or disabled, as well as the configuration of various NUMA memory balancing parameters161718
  • OpenSolaris models NUMA architecture with lgroups
  • FreeBSD added Initial NUMA affinity and policy configuration in version 110 19

See alsoedit

  • Uniform memory access UMA
  • Cluster computing
  • Symmetric multiprocessing SMP
  • Cache only memory architecture COMA
  • Scratchpad memory SPM
  • Partitioned global address space
  • Supercomputer
  • Silicon Graphics SGI
  • HiperDispatch
  • Intel QuickPath Interconnect QPI
  • HyperTransport


  1. ^ a b Nakul Manchanda; Karan Anand 2010-05-04 "Non-Uniform Memory Access NUMA" PDF New York University Retrieved 2014-01-27 
  2. ^ Sergey Blagodurov; Sergey Zhuravlev; Mohammad Dashti; Alexandra Fedorov 2011-05-02 "A Case for NUMA-aware Contention Management on Multicore Systems" PDF Simon Fraser University Retrieved 2014-01-27 
  3. ^ a b Zoltan Majo; Thomas R Gross 2011 "Memory System Performance in a NUMA Multicore Multiprocessor" PDF ACM Retrieved 2014-01-27 
  4. ^ "Intel Dual-Channel DDR Memory Architecture White Paper" PDF Rev 10 ed Infineon Technologies North America and Kingston Technology September 2003 Archived from the original PDF, 1021 KB on 2011-09-29 Retrieved 2007-09-06 
  5. ^ Intel Corp 2008 Intel QuickPath Architecture White paper Retrieved from http://wwwintelcom/pressroom/archive/reference/whitepaper_QuickPathpdf
  6. ^ Intel Corporation September 18th, 2007 Gelsinger Speaks To Intel And High-Tech Industry's Rapid Technology CadenPress release Retrieved from http://wwwintelcom/pressroom/archive/releases/2007/20070918corp_bhtm
  7. ^ "ccNUMA: Cache Coherent Non-Uniform Memory Access" slidesharenet 2014 Retrieved 2014-01-27 
  8. ^ Per Stenstromt; Truman Joe; Anoop Gupta 2002 "Comparative Performance Evaluation of Cache-Coherent NUMA and COMA Architectures" PDF ACM Retrieved 2014-01-27 
  9. ^ David B Gustavson September 1991 "The Scalable Coherent Interface and Related Standards Projects" PDF SLAC Publication 5656 Stanford Linear Accelerator Center Retrieved January 27, 2014 
  10. ^ "The NumaChip enables cache coherent low cost shared memory" Numascalecom Retrieved 2014-01-27 
  11. ^ NUMA Support MSDN
  12. ^ Java HotSpot™ Virtual Machine Performance Enhancements
  13. ^ "Linux Scalability Effort: NUMA Group Homepage" sourceforgenet 2002-11-20 Retrieved 2014-02-06 
  14. ^ "Linux kernel 38, Section 18 Automatic NUMA balancing" kernelnewbiesorg 2013-02-08 Retrieved 2014-02-06 
  15. ^ Jonathan Corbet 2012-11-14 "NUMA in a hurry" LWNnet Retrieved 2014-02-06 
  16. ^ "Linux kernel 313, Section 16 Improved performance in NUMA systems" kernelnewbiesorg 2014-01-19 Retrieved 2014-02-06 
  17. ^ "Linux kernel documentation: Documentation/sysctl/kerneltxt" kernelorg Retrieved 2014-02-06 
  18. ^ Jonathan Corbet 2013-10-01 "NUMA scheduling progress" LWNnet Retrieved 2014-02-06 
  19. ^ "FreeBSD 110-RELEASE Release Notes" freebsdorg 2016-09-22 

This article is based on material taken from the Free On-line Dictionary of Computing prior to 1 November 2008 and incorporated under the "relicensing" terms of the GFDL, version 13 or later

External linksedit

  • Page-based distributed shared memory
  • OpenSolaris NUMA Project
  • Introduction video for the Alpha EV7 system architecture
  • More videos related to EV7 systems: CPU, IO, etc
  • NUMA optimization in Windows Applications
  • NUMA Support in Linux at SGI
  • Intel Tukwila
  • Intel QPI CSI explained
  • current Itanium NUMA systems

Non-uniform memory access Information about

Non-uniform memory access

Non-uniform memory access
Non-uniform memory access

Non-uniform memory access Information Video

Non-uniform memory access viewing the topic.
Non-uniform memory access what, Non-uniform memory access who, Non-uniform memory access explanation

There are excerpts from wikipedia on this article and video

Random Posts

Social Accounts

Facebook Twitter VK
Copyright © 2014. Search Engine