IBM eX5 Implementation Guide

128 IBM eX5 Implementation Guide

The MAX5 is connected to the x3690 X5 using two cables, connecting the QPI ports on the

server to two of the four QPI ports on the MAX5. The other two QPI ports of the MAX5 are

unused. The EXA ports are for future scaling capabilities.

Figure 4-11 shows architecturally how a single-node x3690 X5 is connected to a MAX5.

Figure 4-11 Connectivity of the x3690 X5 with a MAX5 memory expansion unit

As shown in Figure 4-11, the x3690 X5 attaches to the MAX5 using QPI links; you can see

that the eX5 chip set in the MAX5 simultaneously connects to both CPUs in the server.

One benefit of this connectivity is that the MAX5 is able to store a copy of the contents of the

last-level cache of all the CPUs in the server. Therefore, when a CPU requests content stored

in the cache of another CPU, the MAX5 not only has that same data stored in its own cache,

it is able to return the acknowledgement of the snoop

and the data to the requesting CPU in

the same transaction. For more information about QPI links and snooping, see 2.2.4,

“QuickPath Interconnect (QPI)” on page 18.

Connectivity of the MAX5 to the x3690 X5 is described in 4.6, “Scalability” on page 128.

For memory configuration information, see 4.8.3, “MAX5 memory” on page 136. For a

description of the power and fans, see 4.12, “Power supplies” on page 173.

4.6 Scalability

In this section, we describe how the x3690 X5 can be expanded to increase the number of

memory DIMMs.

Tip: The Xeon E6510 processor does not support the use of the MAX5.

QPI

links

16 DIMMs on

system planar

Memory tray

(16 DIMMs)

SMI

links

CPU

0

CPU

1

MAX5

EXA

QPI

QPI

EXA

EXA EXAQPI

x3690 X5

QPIQPI

QPI

QPI

32 DIMMs

on MAX5

External

QPI cables

Chapter 4. IBM System x3690 X5 129

The x3690 X5 supports the following configurations:

򐂰 A single x3690 X5 server with two processor sockets. This configuration is sometimes

referred to as a

single-node server.

򐂰 A single x3690 X5 server with a single MAX5 memory expansion unit attached. This

configuration is sometimes referred to as a

memory-expanded server.

The MAX5 memory expansion unit permits the x3690 X5 to scale to an additional 32 DDR3

DIMM sockets.

Connecting the single-node x3690 X5 with the MAX5 memory expansion unit uses two QPI

cables, part number 59Y6269, as listed in Table 4-4. Figure 4-12 shows the connectivity.

Figure 4-12 Connecting the MAX5 to a single-node x3690 X5

Connecting the MAX5 to a single-node x3690 X5 requires one IBM MAX5 to x3690 X5 Cable

Kit, which consists of two QPI cables. See Table 4-4.

Table 4-4 Ordering information for the IBM MAX5 to x3690 X5 Cable Kit

Two-node configurations: The 2-node configurations (with and without MAX5) are not

supported.

Number of processors: The MAX5 is supported with either one or two processors

installed in the x3690 X5. However, the recommendation is to have two processors

installed and memory installed in every DIMM socket in the server to maximize

performance.

Rack rear

Part number Feature code Description

59Y6269 7481 IBM MAX5 to x3690 X5 Cable Kit (two cables)

Note: There is no setup on the Integrated Management Module (IMM) for scalability.

Ensure that you have updated all firmware before attaching the MAX5, especially the Field

Programmable Gate Array (FPGA). For more information about updating firmware, see

9.10, “Firmware update tools and methods” on page 509.