book

Troubleshooting Cisco Nexus Switches and NX-OS, First Edition

by Vinit Jain, Brad Edgeworth, Richard Furr

June 2018

Intermediate to advanced

1072 pages

35h 44m

English

Cisco Press

Read now

Unlock full access

Nexus Platforms OverviewNexus 2000 SeriesNexus 3000 SeriesNexus 5000 SeriesNexus 6000 SeriesNexus 7000 SeriesNexus 9000 SeriesNX-OS ArchitectureThe KernelSystem Manager (sysmgr)Messages and Transactional ServicesPersistent Storage ServicesFeature ManagerNX-OS Line Card MicrocodeFile SystemsFlash File SystemOnboard Failure LoggingLogflashUnderstanding NX-OS Software Releases and PackagingSoftware Maintenance UpgradesLicensingNX-OS High-Availability InfrastructureSupervisor RedundancyISSUNX-OS Virtualization FeaturesVirtual Device ContextsVirtual Routing and ForwardingVirtual Port ChannelManagement and Operations CapabilitiesNX-OS Advanced CLITechnical Support FilesAccounting LogFeature Event-HistoryDebug Options: Log File and FiltersConfiguration Checkpoint and RollbackConsistency CheckersFeature Scheduler, EEM, and PythonBash ShellSummaryReferences
Packet Capture: Network SnifferEncapsulated Remote SPANSPAN on Latency and DropSPAN-on-LatencySPAN-on-DropNexus Platform ToolsEthanalyzerPacket TracerNetFlowNetFlow ConfigurationEnable NetFlow FeatureDefine a Flow RecordDefine a Flow ExporterDefine and Apply the Flow MonitorNetFlow SamplingsFlowNetwork Time ProtocolEmbedded Event ManagerLoggingDebug LogfilesAccounting LogEvent-HistorySummaryReferences
Troubleshooting Hardware IssuesGeneric Online Diagnostic TestsBootup DiagnosticsRuntime DiagnosticsGOLD Test and EEM SupportNexus Device Health ChecksHardware and Process CrashesPacket LossInterface Errors and DropsPlatform-Specific DropsNexus Fabric ExtendersVirtual Device ContextVDC Resource TemplateConfiguring VDCVDC InitializationOut-of-Band and In-Band ManagementVDC ManagementLine Card Interop LimitationsTroubleshooting NX-OS System ComponentsMessage and Transaction ServicesNetstack and Packet ManagerNetstack TCPUDP ComponentARP and Adjacency ManagerUnicast Forwarding ComponentsUnicast Routing Information BaseUFDM and IPFIBEthPM and Port-ClientHWRL, CoPP, and System QoSMTU SettingsFEX Jumbo MTU SettingsTroubleshooting MTU IssuesSummaryReferences
Network Layer 2 Communication OverviewVirtual LANsVLAN CreationAccess PortsTrunk PortsNative VLANsAllowed VLANsPrivate VLANSIsolated Private VLANsCommunity Private VLANsUsing a Promiscuous PVLAN Port on Switched Virtual InterfaceTrunking PVLANs Between SwitchesSpanning Tree Protocol FundamentalsIEEE 802.1D Spanning Tree ProtocolRapid Spanning Tree ProtocolSpanning-Tree Path CostRoot Bridge ElectionLocating Root PortsLocating Blocked Switch PortsVerification of VLANS on Trunk LinksSpanning Tree Protocol TuningMultiple Spanning-Tree Protocol (MST)MST ConfigurationMST VerificationMST TuningDetecting and Remediating Forwarding LoopsMAC Address NotificationsBPDU GuardBPDU FilterProblems with Unidirectional LinksSpanning Tree Protocol Loop GuardUnidirectional Link DetectionBridge AssuranceSummaryReferences
Port-ChannelsBasic Port-Channel ConfigurationVerifying Port-Channel StatusVerifying LACP PacketsAdvanced LACP Configuration OptionsMinimum Number of Port-Channel Member InterfacesMaximum Number of Port-Channel Member InterfacesLACP System PriorityLACP Interface PriorityLACP FastGraceful ConvergenceSuspend IndividualPort-Channel Member Interface ConsistencyTroubleshooting LACP Interface EstablishmentTroubleshooting Traffic Load-BalancingVirtual Port-ChannelvPC FundamentalsvPC DomainvPC Peer-KeepalivevPC Peer LinkvPC Member LinksvPC Operational BehaviorvPC ConfigurationvPC VerificationVerifying the vPC Domain StatusVerifying the Peer-KeepalivevPC Consistency-CheckerAdvanced vPC FeaturesvPC Orphan PortsvPC AutorecoveryvPC Peer-GatewayvPC ARP SynchronizationBackup Layer 3 RoutingLayer 3 Routing over vPCFabricPathFabricPath Terminologies and ComponentsFabricPath Packet FlowFabricPath ConfigurationFabricPath Verification and TroubleshootingFabricPath DevicesEmulated Switch and vPC+vPC+ ConfigurationvPC+ Verification and TroubleshootingSummaryReferences
IP SLAICMP Echo ProbeUDP Echo ProbeUDP Jitter ProbeTCP Connect ProbeObject TrackingObject Tracking for the InterfaceObject Tracking for Route StateObject Tracking for Track-List StateUsing Track Objects with Static RoutesIPv4 ServicesDHCP RelayDHCP SnoopingDynamic ARP InspectionARP ACLsIP Source GuardUnicast RPFIPv6 ServicesNeighbor DiscoveryIPv6 Address AssignmentDHCPv6 Relay AgentDHCPv6 Relay LDRAIPv6 First-Hop SecurityRA GuardIPv6 SnoopingDHCPv6 GuardFirst-Hop Redundancy ProtocolHSRPHSRPv6VRRPGLBPSummary
EIGRP FundamentalsTopology TablePath Metric CalculationEIGRP CommunicationBaseline EIGRP ConfigurationTroubleshooting EIGRP Neighbor AdjacencyVerification of Active InterfacesPassive InterfaceVerification of EIGRP PacketsConnectivity Must Exist Using the Primary SubnetEIGRP ASN MismatchMismatch K ValuesProblems with Hello and Hold TimersEIGRP Authentication IssuesInterface-Based EIGRP AuthenticationGlobal EIGRP AuthenticationTroubleshooting Path Selection and Missing RoutesLoad BalancingStubMaximum-HopsDistribute ListOffset ListsInterface-Based SettingsRedistributionClassic Metrics vs. Wide MetricsProblems with ConvergenceActive QueryStuck in ActiveSummaryReferences
OSPF FundamentalsInter-Router CommunicationOSPF Hello PacketsNeighbor StatesDesignated RoutersAreasLink State AdvertisementsTroubleshooting OSPF Neighbor AdjacencyBaseline OSPF ConfigurationOSPF Neighbor VerificationConfirmation of OSPF InterfacesPassive InterfaceVerification of OSPF PacketsConnectivity Must Exist Using the Primary SubnetMTU RequirementsUnique Router-IDInterface Area Numbers Must MatchOSPF Stub (Area Flags) Settings Must MatchDR RequirementsTimersAuthenticationTroubleshooting Missing RoutesDiscontiguous NetworkDuplicate Router IDFiltering RoutesRedistributionOSPF Forwarding AddressTroubleshooting OSPF Path SelectionIntra-Area RoutesInter-Area RoutesExternal Route SelectionE1 and N1 External RoutesE2 and N2 External RoutesProblems with Intermixed RFC 1583 and RFC 2328 DevicesInterface Link CostsSummaryReferences
IS-IS FundamentalsAreasNET AddressingInter-Router CommunicationIS Protocol HeaderTLVsIS PDU AddressingIS-IS Hello (IIH) PacketsLink-State PacketsLSP IDAttribute FieldsLSP Packet and TLVsDesignated Intermediate SystemPath SelectionTroubleshooting IS-IS Neighbor AdjacencyBaseline IS-IS ConfigurationIS-IS Neighbor VerificationConfirmation of IS-IS InterfacesPassive InterfaceVerification of IS-IS PacketsConnectivity Must Exist Using the Primary SubnetMTU RequirementsUnique System-IDArea Must Match Between L1 AdjacenciesChecking IS-IS Adjacency CapabilitiesDIS RequirementsIIH AuthenticationTroubleshooting Missing RoutesDuplicate System IDInterface Link CostsMismatch of Metric ModesL1 to L2 Route PropagationsSuboptimal RoutingRedistributionSummaryReferences
Conditional MatchingAccess Control ListsACLs and ACL Manager ComponentInterior Gateway Protocol (IGP) Network SelectionBGP Network SelectionPrefix Matching and Prefix-ListsPrefix MatchingPrefix ListsRoute-MapsConditional MatchingMultiple Conditional Match ConditionsComplex MatchingOptional ActionsIncomplete Configuration of Routing PoliciesDiagnosing Route Policy MangerPolicy-Based RoutingSummaryReferences
BGP FundamentalsAddress FamiliesPath AttributesLoop PreventionBGP SessionsBGP IdentifierBGP MessagesOPENUPDATENOTIFICATIONKEEPALIVEBGP Neighbor StatesIdleConnectActiveOpenSentOpenConfirmEstablishedBGP Configuration and VerificationTroubleshooting BGP Peering IssuesTroubleshooting BGP Peering Down IssuesVerifying ConfigurationVerifying Reachability and Packet LossVerifying ACLs and Firewalls in the PathVerifying TCP SessionsOPEN Message ErrorsBGP DebugsDemystifying BGP NotificationsTroubleshooting IPv6 PeersBGP Peer Flapping IssuesBad BGP UpdateHold Timer ExpiredBGP Keepalive GenerationMTU Mismatch IssuesBGP Route Processing and Route PropagationBGP Route AdvertisementNetwork StatementRedistributionRoute AggregationDefault-Information OriginateBGP Best Path CalculationBGP MultipathEBGP and IBGP MultipathBGP Update Generation ProcessBGP ConvergenceScaling BGPTuning BGP MemoryPrefixesPathsAttributesScaling BGP ConfigurationSoft Reconfiguration Inbound Versus Route RefreshScaling BGP with Route-ReflectorsLoop Prevention in Route ReflectorsMaximum PrefixesBGP Max ASBGP Route Filtering and Route PoliciesPrefix-List-Based FilteringFilter-ListsBGP Route-MapsRegular Expressions (RegEx)_ Underscore^ Caret$ Dollar Sign[ ] Brackets- Hyphen[^] Caret in Brackets( ) Parentheses and | Pipe. Period+ Plus Sign? Question Mark* AsteriskAS-Path Access ListBGP CommunitiesLooking Glass and Route ServersLogs CollectionSummaryFurther ReadingReferences
Bidirectional Forwarding DetectionAsynchronous ModeAsynchronous Mode with Echo FunctionConfiguring and Verifying BFD SessionsNexus High AvailabilityStateful SwitchoverISSUGraceful Insertion and RemovalCustom Maintenance ProfileSummaryReferences
Multicast FundamentalsMulticast TerminologyLayer 2 Multicast AddressesLayer 3 Multicast AddressesNX-OS Multicast ArchitectureReplicationProtecting the Central Processing UnitNX-OS Multicast ImplementationStatic JoinsClearing an MROUTE EntryMulticast Boundary and FilteringEvent-Histories and Show TechsIGMPIGMPv2IGMPv3IGMP SnoopingIGMP VerificationPIM MulticastPIM Protocol State and TreesPIM Message TypesPIM Hello MessagePIM Register MessagePIM Register-Stop MessagePIM Join-Prune MessagePIM Bootstrap MessagePIM Assert MessagePIM Candidate RP Advertisement MessagePIM DF Election MessagePIM Interface and Neighbor VerificationPIM Any Source MulticastPIM ASM ConfigurationPIM ASM VerificationPIM ASM Event-History and MROUTE State VerificationPIM ASM Platform VerificationPIM BidirectionalBiDIR ConfigurationBiDIR VerificationPIM RP ConfigurationStatic RP ConfigurationAuto-RP Configuration and VerificationBSR Configuration and VerificationAnycast-RP Configuration and VerificationAnycast RP with MSDPPIM Anycast RPPIM Source Specific MulticastSSM ConfigurationSSM VerificationMulticast and Virtual Port-ChannelvPC-Connected SourcevPC-Connected ReceivervPC Considerations for Multicast TrafficDuplicate Multicast PacketsReserved VLANEthanalyzer ExamplesSummaryReferences
OTV FundamentalsFlood Control and Broadcast OptimizationSupported OTV PlatformsOTV TerminologyDeploying OTVOTV Deployment ModelsOTV Site VLANOTV ConfigurationUnderstanding and Verifying the OTV Control PlaneOTV Multicast ModeOTV IS-IS Adjacency VerificationOTV IS-IS Topology TableOTV IS-IS AuthenticationAdjacency Server ModeOTV Control Plane Policing (CoPP)Understanding and Verifying the OTV Data PlaneOTV ARP Resolution and ARP-ND-CacheBroadcastsUnknown Unicast FramesOTV Unicast Traffic with a Multicast Enabled TransportOTV Multicast Traffic with a Multicast Enabled TransportOTV Multicast Traffic with a Unicast Transport (Adjacency Server Mode)Advanced OTV FeaturesFirst Hop Routing Protocol LocalizationMultihomingIngress Routing OptimizationVLAN TranslationOTV Tunnel DepolarizationOTV Fast Failure DetectionSummaryReferences
Introduction to Automation and ProgrammabilityIntroduction to Open NX-OSShells and ScriptingBash ShellGuest ShellPythonNX-SDKNX-APISummaryReferences

Overview

The definitive deep-dive guide to hardware and software troubleshooting on Cisco Nexus switches

The Cisco Nexus platform and NX-OS switch operating system combine to deliver unprecedented speed, capacity, resilience, and flexibility in today's data center networks. Troubleshooting Cisco Nexus Switches and NX-OS is your single reference for quickly identifying and solving problems with these business-critical technologies.

Three expert authors draw on deep experience with large Cisco customers, emphasizing the most common issues in real-world deployments, including problems that have caused major data center outages. Their authoritative, hands-on guidance addresses both features and architecture, helping you troubleshoot both control plane forwarding and data plane/data path problems and use NX-OS APIs to automate and simplify troubleshooting. Throughout, you'll find real-world configurations, intuitive illustrations, and practical insights into key platform-specific behaviors.

This is an indispensable technical resource for all Cisco network consultants, system/support engineers, network operations professionals, and CCNP/CCIE certification candidates working in the data center domain.

Understand the NX-OS operating system and its powerful troubleshooting tools
Solve problems with cards, hardware drops, fabrics, and CoPP policies
Troubleshoot network packet switching and forwarding
Properly design, implement, and troubleshoot issues related to Virtual Port Channels (VPC and VPC+)
Optimize routing through filtering or path manipulation
Optimize IP/IPv6 services and FHRP protocols (including HSRP, VRRP, and Anycast HSRP)
Troubleshoot EIGRP, OSPF, and IS-IS neighbor relationships and routing paths
Identify and resolve issues with Nexus route maps
Locate problems with BGP neighbor adjacencies and enhance path selection
Troubleshoot high availability components (BFD, SSO, ISSU, and GIR)
Understand multicast protocols and troubleshooting techniques
Identify and solve problems with OTV
Use NX-OS APIs to automate troubleshooting and administrative tasks