Troubleshooting Cisco Nexus Switches and NX-OS, 1st edition
Published by Cisco Press (May 24, 2018) © 2018
- Vinit Jain Best-selling Cisco Press author, expert trainer, and Cisco Certified Internetwork Expert (No. 22854)
- Brad Edgeworth Best-selling Cisco Press author, expert trainer, and Cisco Certified Internetwork Expert (No. 31574)
- Richard Furr
eTextbook
- Available for purchase from all major ebook resellers, including InformIT.com.
- To request a review copy, click on the "Request a Review Copy" button.
- A print text (hardcover or paperback)Â
- Free shipping
- Also available for purchase as an ebook from all major ebook resellers, including InformIT.com
The Cisco Nexus platform and NX-OS switch operating system combine to deliver unprecedented speed, capacity, resilience, and flexibility in today's data center networks. Troubleshooting Cisco Nexus Switches and NX-OS is your single reference for quickly identifying and solving problems with these business-critical technologies.
Three expert authors draw on deep experience with large Cisco customers, emphasizing the most common issues in real-world deployments, including problems that have caused major data center outages. Their authoritative, hands-on guidance addresses both features and architecture, helping you troubleshoot both control plane forwarding and data plane/data path problems and use NX-OS APIs to automate and simplify troubleshooting. Throughout, you'll find real-world configurations, intuitive illustrations, and practical insights into key platform-specific behaviors.
This is an indispensable technical resource for all Cisco network consultants, system/support engineers, network operations professionals, and CCNP/CCIE certification candidates working in the data center domain.
- Troubleshoot data center deployments with NX-OS
- Master techniques for minimizing network outages and increasing network performance and scalability
- Gain an in-depth understanding of Nexus and NX-OS architecture at the packet level to become a more effective troubleshooter
- Learn detailed troubleshooting techniques for unicast and multicast forwarding issues
- Master key concepts and techniques for troubleshooting at Layer 2, including FabricPath, vPC and vPC+
The full text downloaded to your computer
With eBooks you can:
- search for key concepts, words and phrases
- make highlights and notes as you study
- share your notes with friends
eBooks are downloaded to your computer and accessible either offline through the Bookshelf (available as a free download), available online and also via the iPad and Android apps.
Upon purchase, you'll gain instant access to this eBook.
Foreword xxvi
Introduction xxvii
Part I Introduction to Troubleshooting Nexus Switches
Chapter 1 Introduction to Nexus Operating System (NX-OS) 1
Nexus Platforms Overview 2
   Nexus 2000 Series 2
   Nexus 3000 Series 3
   Nexus 5000 Series 4
   Nexus 6000 Series 4
   Nexus 7000 Series 5
   Nexus 9000 Series 6
NX-OS Architecture 8
   The Kernel 9
   System Manager (sysmgr) 9
   Messages and Transactional Services 11
   Persistent Storage Services 13
   Feature Manager 14
   NX-OS Line Card Microcode 17
   File Systems 19
Understanding NX-OS Software Releases and Packaging 25
   Software Maintenance Upgrades 27
   Licensing 28
NX-OS High-Availability Infrastructure 28
   Supervisor Redundancy 29
   ISSU 34
NX-OS Virtualization Features 35
   Virtual Device Contexts 35
   Virtual Routing and Forwarding 37
   Virtual Port Channel 37
Management and Operations Capabilities 39
   NX-OS Advanced CLI 39
   Technical Support Files 44
   Accounting Log 45
   Feature Event-History 46
   Debug Options: Log File and Filters 47
   Configuration Checkpoint and Rollback 48
   Consistency Checkers 49
   Feature Scheduler, EEM, and Python 50
   Bash Shell 51
Summary 51
References 51
Chapter 2 NX-OS Troubleshooting Tools 53
Packet Capture: Network Sniffer 53
   Encapsulated Remote SPAN 57
   SPAN on Latency and Drop 60
Nexus Platform Tools 63
   Ethanalyzer 63
   Packet Tracer 71
NetFlow 72
   NetFlow Configuration 73
   NetFlow Sampling 77
   sFlow 78
Network Time Protocol 81
Embedded Event Manager 83
Logging 87
   Debug Logfiles 90
   Accounting Log 91
   Event-History 92
Summary 93
References 93
Chapter 3 Troubleshooting Nexus Platform Issues 95
Troubleshooting Hardware Issues 95
   Generic Online Diagnostic Tests 98
   Nexus Device Health Checks 108
   Nexus Fabric Extenders 124
Virtual Device Context 130
   VDC Resource Template 131
   Configuring VDC 133
   VDC Initialization 134
   Out-of-Band and In-Band Management 137
   VDC Management 137
Troubleshooting NX-OS System Components 142
   Message and Transaction Services 144
   Netstack and Packet Manager 148
   ARP and Adjacency Manager 160
   EthPM and Port-Client 175
HWRL, CoPP, and System QoS 179
   MTU Settings 192
Summary 195
References 196
Part II Troubleshooting Layer 2 Forwarding
Chapter 4 Nexus Switching 197
Network Layer 2 Communication Overview 197
Virtual LANs 200
   VLAN Creation 201
   Access Ports 203
   Trunk Ports 204
   Private VLANS 207
Spanning Tree Protocol Fundamentals 218
   IEEE 802.1D Spanning Tree Protocol 219
   Rapid Spanning Tree Protocol 220
   Multiple Spanning-Tree Protocol (MST) 236
Detecting and Remediating Forwarding Loops 241
   MAC Address Notifications 242
   BPDU Guard 243
   BPDU Filter 244
   Problems with Unidirectional Links 245
Summary 252
References 254
Chapter 5 Port-Channels, Virtual Port-Channels, and FabricPath 255
Port-Channels 255
   Basic Port-Channel Configuration 259
   Verifying Port-Channel Status 260
   Verifying LACP Packets 262
   Advanced LACP Configuration Options 265
   LACP System Priority 268
   Port-Channel Member Interface Consistency 271
   Troubleshooting LACP Interface Establishment 272
   Troubleshooting Traffic Load-Balancing 272
Virtual Port-Channel 274
   vPC Fundamentals 275
   vPC Configuration 278
   vPC Verification 280
   Advanced vPC Features 288
FabricPath 294
   FabricPath Terminologies and Components 296
   FabricPath Packet Flow 297
   FabricPath Configuration 300
   FabricPath Verification and Troubleshooting 303
   FabricPath Devices 310
Emulated Switch and vPC+ 310
   vPC+ Configuration 311
   vPC+ Verification and Troubleshooting 314
Summary 320
References 320
Part III Troubleshooting Layer 3 Routing
Chapter 6 Troubleshooting IP and IPv6 Services 321
IP SLA 321
   ICMP Echo Probe 322
   UDP Echo Probe 324
   UDP Jitter Probe 325
   TCP Connect Probe 328
Object Tracking 329
   Object Tracking for the Interface 330
   Object Tracking for Route State 330
   Object Tracking for Track-List State 332
   Using Track Objects with Static Routes 334
IPv4 Services 335
   DHCP Relay 335
   DHCP Snooping 341
   Dynamic ARP Inspection 345
   IP Source Guard 349
   Unicast RPF 351
IPv6 Services 352
   Neighbor Discovery 352
   IPv6 Address Assignment 357
   IPv6 First-Hop Security 362
First-Hop Redundancy Protocol 370
   HSRP 370
   VRRP 380
   GLBP 385
Summary 391
Chapter 7 Troubleshooting Enhanced Interior Gateway Routing Protocol (EIGRP) 393
EIGRP Fundamentals 393
   Topology Table 395
   Path Metric Calculation 396
   EIGRP Communication 399
   Baseline EIGRP Configuration 399
Troubleshooting EIGRP Neighbor Adjacency 401
   Verification of Active Interfaces 402
   Passive Interface 403
   Verification of EIGRP Packets 405
   Connectivity Must Exist Using the Primary Subnet 409
   EIGRP ASN Mismatch 412
   Mismatch K Values 413
   Problems with Hello and Hold Timers 414
   EIGRP Authentication Issues 416
Troubleshooting Path Selection and Missing Routes 419
   Load Balancing 421
   Stub 421
   Maximum-Hops 424
   Distribute List 426
   Offset Lists 427
   Interface-Based Settings 430
   Redistribution 430
   Classic Metrics vs. Wide Metrics 433
Problems with Convergence 439
   Active Query 441
   Stuck in Active 443
Summary 446
References 447
Chapter 8 Troubleshooting Open Shortest Path First (OSPF) 449
OSPF Fundamentals 449
   Inter-Router Communication 450
   OSPF Hello Packets 450
   Neighbor States 451
   Designated Routers 452
   Areas 453
   Link State Advertisements 453
Troubleshooting OSPF Neighbor Adjacency 456
   Baseline OSPF Configuration 456
   OSPF Neighbor Verification 458
   Confirmation of OSPF Interfaces 460
   Passive Interface 461
   Verification of OSPF Packets 463
   Connectivity Must Exist Using the Primary Subnet 468
   MTU Requirements 469
   Unique Router-ID 471
   Interface Area Numbers Must Match 471
   OSPF Stub (Area Flags) Settings Must Match 473
   DR Requirements 474
   Timers 476
   Authentication 478
Troubleshooting Missing Routes 482
   Discontiguous Network 482
   Duplicate Router ID 485
   Filtering Routes 487
   Redistribution 487
   OSPF Forwarding Address 488
Troubleshooting OSPF Path Selection 494
   Intra-Area Routes 494
   Inter-Area Routes 495
   External Route Selection 495
   E1 and N1 External Routes 496
   E2 and N2 External Routes 497
   Problems with Intermixed RFC 1583 and RFC 2328 Devices 499
   Interface Link Costs 500
Summary 504
References 505
Chapter 9 Troubleshooting Intermediate System-Intermediate System (IS-IS) 507
IS-IS Fundamentals 507
   Areas 508
   NET Addressing 509
   Inter-Router Communication 511
   IS Protocol Header 511
   TLVs 512
   IS PDU Addressing 512
   IS-IS Hello (IIH) Packets 513
   Link-State Packets 515
   Designated Intermediate System 516
   Path Selection 517
Troubleshooting IS-IS Neighbor Adjacency 518
   Baseline IS-IS Configuration 518
   IS-IS Neighbor Verification 520
   Confirmation of IS-IS Interfaces 523
   Passive Interface 526
   Verification of IS-IS Packets 528
   Connectivity Must Exist Using the Primary Subnet 535
   MTU Requirements 537
   Unique System-ID 539
   Area Must Match Between L1 Adjacencies 539
   Checking IS-IS Adjacency Capabilities 541
   DIS Requirements 543
   IIH Authentication 544
Troubleshooting Missing Routes 546
   Duplicate System ID 546
   Interface Link Costs 549
   Mismatch of Metric Modes 553
   L1 to L2 Route Propagations 556
   Suboptimal Routing 562
   Redistribution 566
Summary 567
References 568
Chapter 10 Troubleshooting Nexus Route-Maps 569
Conditional Matching 569
   Access Control Lists 569
   ACLs and ACL Manager Component 570
   Prefix Matching and Prefix-Lists 577
Route-Maps 581
   Conditional Matching 582
   Optional Actions 586
   Incomplete Configuration of Routing Policies 586
   Diagnosing Route Policy Manger 586
Policy-Based Routing 591
Summary 594
References 595
Chapter 11 Troubleshooting BGP 597
BGP Fundamentals 597
   Address Families 598
   Path Attributes 599
   Loop Prevention 599
BGP Sessions 600
   BGP Identifier 601
   BGP Messages 601
   BGP Neighbor States 602
   BGP Configuration and Verification 605
Troubleshooting BGP Peering Issues 609
   Troubleshooting BGP Peering Down Issues 609
   Demystifying BGP Notifications 619
   Troubleshooting IPv6 Peers 621
   BGP Peer Flapping Issues 622
BGP Route Processing and Route Propagation 630
   BGP Route Advertisement 631
   BGP Best Path Calculation 636
   BGP Multipath 640
   BGP Update Generation Process 643
   BGP Convergence 646
Scaling BGP 649
   Tuning BGP Memory 650
   Soft Reconfiguration Inbound Versus Route Refresh 654
   Scaling BGP with Route-Reflectors 657
   Maximum Prefixes 659
   BGP Max AS 662
BGP Route Filtering and Route Policies 662
   Prefix-List-Based Filtering 663
   Filter-Lists 669
BGP Route-Maps 673
   Regular Expressions (RegEx) 676
   AS-Path Access List 684
   BGP Communities 684
Looking Glass and Route Servers 687
Logs Collection 687
Summary 687
Further Reading 688
References 688
Part IV Troubleshooting High Availability
Chapter 12 High Availability 689
Bidirectional Forwarding Detection 689
   Asynchronous Mode 691
   Asynchronous Mode with Echo Function 693
   Configuring and Verifying BFD Sessions 693
Nexus High Availability 707
   Stateful Switchover 707
   ISSU 713
Graceful Insertion and Removal 719
   Custom Maintenance Profile 727
Summary 731
References 732
Part V Multicast Network Traffic
Chapter 13 Troubleshooting Multicast 733
Multicast Fundamentals 734
   Multicast Terminology 735
   Layer 2 Multicast Addresses 738
   Layer 3 Multicast Addresses 739
NX-OS Multicast Architecture 741
   Replication 744
   Protecting the Central Processing Unit 745
   NX-OS Multicast Implementation 747
IGMP 750
   IGMPv2 751
   IGMPv3 752
   IGMP Snooping 756
   IGMP Verification 761
PIM Multicast 771
   PIM Protocol State and Trees 772
   PIM Message Types 773
   PIM Interface and Neighbor Verification 780
   PIM Any Source Multicast 785
   PIM Bidirectional 799
   PIM RP Configuration 811
   PIM Source Specific Multicast 841
Multicast and Virtual Port-Channel 848
   vPC-Connected Source 849
   vPC-Connected Receiver 861
   vPC Considerations for Multicast Traffic 870
Ethanalyzer Examples 871
Summary 871
References 872
Part VI Troubleshooting Nexus Tunneling
Chapter 14 Troubleshooting Overlay Transport Virtualization (OTV) 875
OTV Fundamentals 875
   Flood Control and Broadcast Optimization 877
   Supported OTV Platforms 878
   OTV Terminology 878
   Deploying OTV 881
Understanding and Verifying the OTV Control Plane 885
   OTV Multicast Mode 887
   OTV IS-IS Adjacency Verification 888
   OTV IS-IS Topology Table 898
   OTV IS-IS Authentication 905
   Adjacency Server Mode 907
   OTV Control Plane Policing (CoPP) 912
Understanding and Verifying the OTV Data Plane 913
   OTV ARP Resolution and ARP-ND-Cache 915
   Broadcasts 917
   Unknown Unicast Frames 918
   OTV Unicast Traffic with a Multicast Enabled Transport 919
   OTV Multicast Traffic with a Multicast Enabled Transport 924
   OTV Multicast Traffic with a Unicast Transport (Adjacency Server Mode) 932
Advanced OTV Features 937
   First Hop Routing Protocol Localization 938
   Multihoming 939
   Ingress Routing Optimization 940
   VLAN Translation 941
   OTV Tunnel Depolarization 942
   OTV Fast Failure Detection 944
Summary 946
References 947
Part VII Network Programmability
Chapter 15 Programmability and Automation 949
Introduction to Automation and Programmability 949
Introduction to Open NX-OS 950
   Shells and Scripting 951
NX-SDK 964
NX-API 968
Summary 975
References 975
Â
9781587145056Â Â Â TOCÂ Â Â 4/24/2018
Â
Vinit Jain, CCIE No. 22854 (R&S, SP, Security & DC), is a technical leader with the Cisco Technical Assistance Center (TAC) providing escalation support in areas of routing and data center technologies. Vinit is a speaker at various networking forums, including Cisco Live events globally on various topics. Prior to joining Cisco, Vinit worked as a CCIE trainer and a network consultant. In addition to his CCIEs, Vinit holds multiple certifications on programming and databases. Vinit graduated from Delhi University in Mathematics and earned his Master's in nformation Technology from Kuvempu University in India. Vinit can be found on Twitter as @VinuGenie.
Brad Edgeworth, CCIE No. 31574 (R&S & SP), is a systems engineer at Cisco Systems. Brad is a distinguished speaker at Cisco Live, where he has presented on various topics. Before joining Cisco, Brad worked as a network architect and consultant for various Fortune 500 companies. Brad’s expertise is based on enterprise and service provider environments with an emphasis on architectural and operational simplicity. Brad holds a Bachelor of Arts degree in Computer Systems Management from St. Edward’s University in Austin, Texas. Brad can be found on Twitter as @BradEdgeworth.
Richard Furr, CCIE No. 9173 (R&S & SP), is a technical leader with the Cisco Technical Assistance Center (TAC), supporting customers and TAC teams around the world. For the past 17 years, Richard has worked for the Cisco TAC and High Touch Technical Support (HTTS) organizations, supporting service provider, enterprise, and data center environments. Richard specializes in resolving complex problems found with routing protocols, MPLS, multicast, and network overlay technologies.
Need help? Get in touch
![Video](https://img.youtube.com/vi/TCIok0KnHDc/hqdefault.jpg)
Pearson eTextbook: What’s on the inside just might surprise you
They say you can’t judge a book by its cover. It’s the same with your students. Meet each one right where they are with an engaging, interactive, personalized learning experience that goes beyond the textbook to fit any schedule, any budget, and any lifestyle.Â
![](https://www.pearson.com/content/dam/global-store/global/plp-pdp/1600x800-GettyImages-1172587378.jpg)
Digital Learning NOW
Extend your professional development and meet your students where they are with free weekly Digital Learning NOW webinars. Attend live, watch on-demand, or listen at your leisure to expand your teaching strategies. Earn digital professional development badges for attending a live session.