84 IBM eX5 Implementation Guide
MAX5 memory as seen by the operating system
MAX5 is capable of two modes of operation in terms of the way that memory is presented to
the operating system:
Memory in MAX5 can be split and assigned between the CPUs on the host system
(partitioned mode). This mode is the default.
Memory in MAX5 can be presented as a pool of space that is not assigned to any
particular CPU (pooled mode).
By default, MAX5 is set to operate in partitioned mode because certain operating systems
behave unpredictably when presented with a pool of memory space. Linux can work with
memory that is presented either as a pool or pre-assigned between CPUs, however for
performance reasons, if you are running Linux, change the setting to pooled mode.
You can change this default setting in UEFI.
3.8.3 Maximizing memory performance
In a single node x3850 X5 that is populated with four CPUs and eight memory cards, there
are a total of 16 memory buffers, as shown in the system block diagram in Figure 3-7 on
page 66. Memory buffers are listed as MB1 and MB2 on each of eight memory cards in that
diagram. Each memory buffer has two memory channels, and each channel can have a
maximum of two
DIMMs per channel (DPC). A single-node x3850 X5 has the following
maximums:
Memory cards: 8
Memory buffers: 16
Memory channels: 32
Number of DIMMs: 64
The x3850 X5 supports a variety of ways to install memory DIMMs in the eight memory cards.
However, it is important to understand that because of the layout of the SMI links, memory
buffers, and memory channels, you must install the DIMMs in the correct locations to
maximize performance.
9 27 and 30
10
10 and 15
11
2 and 7
12
19 and 22
13
25 and 32
14
12 and 13
15
4 and 5
16
17 and 24
VMware vSphere support: MAX5 requires VMware vSphere 4.1 or later.
DIMM pair DIMM slots
Chapter 3. IBM System x3850 X5 and x3950 X5 85
Figure 3-19 shows eight possible memory configurations for the two memory cards and 16
DIMMs connected to one processor socket. Each configuration has a relative performance
score. Note the key information from this chart:
The best performance is achieved by populating all memory DIMMs in two memory cards
for each processor installed (configuration 1).
Populating only one memory card per socket can result in approximately a 50%
performance degradation (compare configuration 1 with 5).
Memory performance is better if you install DIMMs on all memory channels than if you
leave any memory channels empty (compare configuration 2 with 3).
Two DIMMs per channel result in better performance that one DIMM per channel
(compare configuration 1 with 2, and compare configuration 5 with 6).
Figure 3-19 Relative memory performance based on DIMM placement (one processor and two memory cards shown)
Use the following general memory population rules:
DIMMs must be installed in matching pairs.
Each memory card requires at least two DIMMs.
Each processor and memory card must have identical amounts of RAM.
Install and populate two memory cards per processor or you can lose memory bandwidth.
1
Each processor:
2 memory controllers
2 DIMMs per channel
8 DIMMs per MC
Mem Ctrl 1 Mem Ctrl 2
1.0
2
Mem Ctrl 1 Mem Ctrl 2
Each processor:
2 memory controllers
1 DIMM per channel
4 DIMMs per MC
0.94
Mem Ctrl 1
Memory card
DIMMs
Channel
Memory buffer
SMI link
Memory controller
3
Mem Ctrl 1 Mem Ctrl 2
Each processor:
2 memory controllers
2 DIMMs per channel
4 DIMMs per MC
0.61
Relative
performance
4
Mem Ctrl 1 Mem Ctrl 2
Each processor:
2 memory controllers
1 DIMM per channel
2 DIMMs per MC
0.58
5
Mem Ctrl 1 Mem Ctrl 2
Each processor:
1 memory controller
2 DIMMs per channel
8 DIMMs per MC
0.51
6
Mem Ctrl 1 Mem Ctrl 2
Each processor:
1 memory controller
1 DIMM per channel
4 DIMMs per MC
0.47
7
Mem Ctrl 1 Mem Ctrl 2
Each processor:
1 memory controller
2 DIMMs per channel
4 DIMMs per MC
0.31
8
Mem Ctrl 1 Mem Ctrl 2
Each processor:
1 memory controller
1 DIMM per channel
2 DIMMs per MC
0.29
1
0.94
0.61
0.51
0.47
0.31
0.29
0.58
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
12345678
Configuration
Relative memory performance
Memory configurations
Get IBM eX5 Implementation Guide now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
