IBM eX5 Implementation Guide

104 IBM eX5 Implementation Guide

򐂰 Slots 5 - 7, the onboard Broadcom-based Ethernet dual-port chip and the custom slot for

the RAID controller are on the first PCIe bridge and require that either CPU 1 or 2 is

installed and operational.

򐂰 Slots 1 - 4 are on the second PCIe bridge and require that either CPU 3 or 4 is installed

and operational.

Table 3-31 shows the order in which to add cards to balance bandwidth between the two PCIe

controllers. However, this installation order assumes that the cards are installed in matched

pairs, or that they have similar throughput capabilities.

Table 3-31 Order for adding cards

Two additional power connectors, one 2x4 and one 2x3, are provided on the planar for

high-power adapters, such as graphics cards. If there is a requirement to use an x16 PCIe

card that is not shown as supported in ServerProven, initiate the SPORE process.

To determine whether a vendor has qualified any x16 cards with the x3850 X5, see IBM

ServerProven at the following website:

http://www.ibm.com/servers/eserver/serverproven/compat/us/serverproven

If the preferred vendor’s logo is displayed, click it to assess options that the vendor has

qualified on the x3850 X5. You can obtain the support caveats for third-party options in

3.12.2, “Optional adapters” on page 107.

In a 2-node configuration, all PCIe slots are available to the operating system that is running

on the complex. They appear as devices on separate PCIe buses.

3.12 I/O cards

This section describes the I/O cards that are suitable for the x3850 X5.

3.12.1 Standard Emulex 10Gb Ethernet Adapter

As described in 3.3, “Models” on page 64, most models include the Emulex 10Gb Ethernet

Adapter as standard. The card is installed in PCIe slot 7. Slot 7 is a nonstandard x8 slot,

which is slightly longer than normal, and it is shown in Figure 3-26 on page 105.

Installation order PCIe slot Slot width Slot bandwidth

a

a. This column correctly shows bandwidth expressed as GB for gigabyte or Gb for gigabit. 10

bits of traffic correspond to 1 byte of data due to the 8:10 encoding scheme. A single PCIe

2.0 lane provides a unidirectional bandwidth of 500 MBps or 5 Gbps.

1 1 x16 PCIe slot 8 GBps (80 Gbps)

2 5 x8 PCIe slot 4 GBps (40 Gbps)

3 3 x8 PCIe slot 4 GBps (40 Gbps)

4 6 x8 PCIe slot 4 GBps (40 Gbps)

5 4 x8 PCIe slot 4 GBps (40 Gbps)

6 7 x8 PCIe slot 4 GBps (40 Gbps)

7 2 x4 PCIe slot 2 GBps (20 Gbps)

Chapter 3. IBM System x3850 X5 and x3950 X5 105

Figure 3-26 Top view of slot 6 and 7 showing that slot 7 is slightly longer than slot 6

The Emulex 10Gb Ethernet Adapter in the x3850 X5 has been customized with a special type

of connector called an

extended edge connector. The card itself is colored blue instead of

green to indicate that it is nonstandard and cannot be installed in a standard x8 PCIe slot.

At the time of writing, only the x3850 X5 and the x3690 X5 have slots that are compatible with

the custom-built Emulex 10Gb Ethernet Adapter that is shown in Figure 3-27.

Figure 3-27 The Emulex 10Gb Ethernet Adapter has a blue circuit board and a longer connector

The Emulex 10Gb Ethernet Adapter is a customer-replaceable unit (CRU). To replace the

adapter (for example, under warranty), order the CRU number, as shown in Table 3-32 on

page 106. The table also shows the regular Emulex 10Gb Virtual Fabric Adapter (VFA) for

IBM System x option, which differs only in the connector type (standard x8) and color of the

circuit board (green).

Tip: The Emulex 10Gb Ethernet Adapter that is standard with most models is a custom

version of the Emulex 10Gb Virtual Fabric Adapter for IBM System x, part number

49Y4250. However, the features and functions of the two adapters are identical.

At 106

pins, slot 7

is slightly

longer than

a standard

x8 PCIe

slot