Cisco Nexus 6000 and 5600 with Fabric Extender 2000 Switch Architecture BRKARC-3452
Nicolas Delecroix Technical Marketing Engineer Data Center Group – Nexus Business Unit
[email protected]
Nexus 5000 Evolution
Nexus 5020 56-Port, 2RU Customers
Nexus 5010 28-Port, 1RU BRKARC-3452
Nexus 5548 48-Port, 1RU
Nexus 5596T 10Gbase-T, 2RU
Nexus 5596 96-Port, 2RU © 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
3
Introducing Cisco Nexus 5600/6000
Nexus 6004 Nexus 5020 56-Port, 2RU Customers
Nexus 5010 28-Port, 1RU BRKARC-3452
Nexus 5548 48-Port, 1RU
Nexus 5596T 10Gbase-T, 2RU
Nexus 5596 96-Port, 2RU © 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
4
96 port 40G, 4RU
Introducing Cisco Nexus 5600/6000 Nexus 5672 48 port 10G, 6 port 40G, 1RU
Nexus 56128 Nexus 6004 96 port 10G, 8 port 40G, 2RU Nexus 5020 56-Port, 2RU Customers
Nexus 5010 28-Port, 1RU BRKARC-3452
Nexus 5548 48-Port, 1RU
Nexus 5596T 10Gbase-T, 2RU
Nexus 5596 96-Port, 2RU © 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
5
96 port 40G, 4RU
Cisco Nexus 6000 and 5600 with Fabric Extender 2000 Switch Architecture BRKARC-3452
Nicolas Delecroix Technical Marketing Engineer Data Center Group – Nexus Business Unit
[email protected]
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
7
Nexus 5672 Chassis Port-Side View • All ports provide Ethernet and FCoE • 16 Unified Ports (orange) also provide 2/4/8G FC • L2 and L3 VXLAN
48 fixed 1/10G SFP+ interfaces BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
8
6 fixed 40G QSFP interfaces
Cisco Nexus 5672 Chassis Power Supply and Fans Power Supply 1+1 Redundancy
Fan Module 2+1 Redundancy
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Console Mgmt0 USB
9
Nexus 56128 Chassis Port-Side View • 48 Fixed 10G SFP+ Ports, 4 fixed 40G QSFP+ ports
• 2 Expansion Slots • Expansion module provides 24 Unified ports 10G Ethernet/FCoE or 2/4/8G FC, and 2 ports 40G Ethernet/FCoE
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
10
Cisco Nexus 56128 Chassis Power Supply and Fans Power Supply 2+2 Redundancy
Fan Module 3+1 Redundancy
Console Mgmt0 USB
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
11
Cisco Nexus 6004 Chassis 12 QSFP+ ports Expansion Module N6K-6004-M12Q
Port-Side View
4RU
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
12
Cisco Nexus 6004 Chassis Power Supply and Fans • Six power supply slots; a minimum of three is required. Supports both 3 + 1 and 3 + 3 redundancy. • Each power supply is rated 1100W, 90—240 VAC. • The chassis has four fan trays. A minimum of three is required.
Power Supply 3 + 3 Grid Redundancy or 3 + 1 Redundancy
Console Mgmt0 USB
Fan Module 3+1 Redundancy BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
13
Unified Port Line-Card Expansion Module - N6004X-M20UP For Nexus 6004 • 20 * 10G SFP+ unified ports expansion module • 2/4/8G FC port • 1G/10G and FCoE • Max number of native FC ports per chassis is 48 (as of NXOS 7.0(2)N1(1) release) • SFP+ port allows support for a larger variety of optical transceivers
• Same transceivers supported as on 5600 10G ports – including 10G long distance, 1G copper and optical BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
14
Cisco Nexus 5600/6000 Airflow • The units have a “port side” and a “fan side”. No front or back • The units support both port side exhaust and port side intake – Port side exhaust (used for in server rack airflow alignment) – Port side intake (used for network rack airflow alignment)
• Different power supply and fan modules are required for different airflow directions
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
15
N5600/6000 and N5500 Comparison Feature
Nexus 5600/6000
Nexus 5500
L2 Throughput
Line rate
Line rate
L3 Throughput
Line rate
160Gbps
Latency
~1us
1.8us
Native 40GE Support
Yes
4xQSFP module – 10G flows
1GE Support
Yes
Yes
Unified Ports FEX Scaling
Yes 48 (6004 L2), 24 (6004 L3, 5600)
Yes 24 with L2, 16 with L3
MAC Table IPv4 Unicast Routes
256K (shared with ARP/ND) 32K
32K 16K
IPv4 Multicast Routes
32K
8K (shared with ARP)
IPv4 ARP/Host Route Entries
128K
16K (shared with IPv6 ND and mroute)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
16
N5600/6000 and N5500 Comparison (cont.) Feature
Nexus 5600/6000
Nexus 5500
IPv6 Unicast Routes
8K (shared with IPv4)
8K (shared with IPv4)
IPv6 ND Entry/Host Route Entries
85K (shared with MAC and ARP)
8K (shared with mroute and ARP)
IPv6 Multicast Routes
32K
8K (shared with ARP, ND. Limit is with /64 routes)
VLAN
4K
4K
VRF
4K
1K
Segment ID
Yes
No
ACL
4K
4K
L4ops for ACL
24 total (16 for UDP/TCP, 8 for TCP flag)
8 for UDP and 8 for TCP
Packet Buffer
640KB per 10G port
vPC, FabricPath
25MB per 3xQSFP (or 12xSFP) ports 2.1MB per 10G port Yes
Dynamic Fabric Automation
Yes
No
VXLAN
Yes*
No
*Software roadmap feature BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
17
Yes
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
18
Nexus 2000 100M/1G Fabric Extender Portfolio
Nexus 2224TP
Nexus 2248TP
Nexus 2248TP-E
2 x 10G SFP+
4 x 10G SFP+
4 x 10G SFP+
Host Facing Ports
24 x 100/1000BASE-T RJ45
48 x 100/1000BASE-T RJ45
48 x 100/1000BASE-T RJ45
Oversubscription
1 : 1.2
1 : 1.2
1 : 1.2
FCoE
N/A
N/A
N/A
Buffer
3.8MB
7.6MB
32MB
No
No
Yes
Uplink Ports
Shared Buffer
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
19
Nexus 2000 1G/10G Fabric Extender Portfolio
Nexus 2232PP
Nexus 2232TM
Nexus 2232TM-E
Nexus 2248PQ
8 x 10G SFP+
8 x 10G SFP+
8 x 10G SFP+
4 x 40G QSFP+
Host Facing Ports
32 x SFP/SFP+ (1/10G)
32 x 1/10GBASE-T RJ45
32 x 1/10GBASE-T RJ45
48 x SFP/SFP+ (1/10G)
Oversubscription
1:4
1:4
1:4
1:3
FCoE
Yes
No
Yes (30m / Cat6a/7)
Yes
Buffer
10.2MB
10.2MB
10.2MB
32MB
No
No
No
Yes
Uplink Ports
Shared Buffer
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
20
Nexus 2000 Fabric Extender Scalability • Nexus 5600 and 6000 support all types of FEX, including HP, Fujitsu, Dell, IBM FEX for blade servers. Adapter-FEX and VM-FEX are also supported.
Cisco Nexus 2000 FEX
48xFEX
Cisco Nexus 5600/6000
+ Nexus 5000
BRKARC-3452
Nexus 5500
Nexus 5600
Nexus 6004
Nexus 7000
Nexus 7700
L2
12
24
24
48
48
64
L3
x
16
24
24
48
64
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
21
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
22
Nexus 5672 - 72 x 10G Unicast Throughput 100
90 80 70 60
Throughput (% of line rate)
50 40 30 20 10 0
RFC 2544 Unicast L2 Port Pair RFC 2544 Unicast L3 Port Pair
64 100 100
80 100 100
96 100 100
128 100 100
256 100 100
512 100 100
1000 100 100
1024 100 100
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
23
1280 100 100
1518 100 100
2240 100 100
4096 100 100
9100 100 100
Nexus 5672 - 72 x 10G Multicast Throughput 100 90 80 70
Throughput (% of line rate)
60 50 40 30 20 10 0
RFC 3918 Mcast L2 One to Many Accumulated RFC 3918 Mcast L3 One to Many Accumulated RFC 3918 Mcast L2 One to Many Distributed RFC 3918 Mcast L3 One to Many Distributed
64 100 100 100 100
80 100 100 100 100
96 100 100 100 100
128 100 100 100 100
256 100 100 100 100
512 100 100 100 100
1000 100 100 100 100
1024 100 100 100 100
1280 100 100 100 100
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
24
1518 100 100 100 100
2240 100 100 100 100
4096 100 100 100 100
9100 100 100 100 100
Nexus 5672 – 72 x 10G Unicast 100% Load Latency 1600 1400 1200 1000 Average Latency (Nano Seconds)
800 600 400 200 0
RFC 2544 unicast L2 port pair RFC 2544 unicast L3 port pair
64 1001 937
128 1021 963
256 1091 1029
512 1111 1048
1024 1105 1047
1280 1104 1046
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
25
1518 1104 1045
2240 1103 1044
4096 1101 1043
9216 1098 1040
Nexus 6004 Performance – Fully Loaded Switch
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
26
Nexus 6004 Performance – Spirent Traffic Generator
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
27
Nexus 6004 - 96 x 40G Unicast Throughput 100 90 80 70 60
Throughput (% of line rate)
50 40 30 20 10 0
RFC 2544 Unicast L2 Port Pair RFC 2544 Unicast L3 Port Pair
64 100 100
80 100 100
96 100 100
128 100 100
256 100 100
512 100 100
1000 100 100
1024 100 100
1280 100 100
1518 100 100
Packet Size (Bytes)
Spirent third party performance report for Nexus 6004 BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
28
2240 100 100
4096 100 100
9100 100 100
Nexus 6004 - 96 x 40G Multicast Throughput
Throughput (% of line rate)
100 90 80 70 60 50 40 30 20 10 0
RFC 3918 Mcast L2 One to Many Accumulated RFC 3918 Mcast L3 One to Many Accumulated RFC 3918 Mcast L2 One to Many Distributed RFC 3918 Mcast L3 One to Many Distributed
64 100 100 100 100
80 100 100 100 100
96 100 100 100 100
128 100 100 100 100
256 100 100 100 100
512 100 100 100 100
1000 100 100 100 100
1024 100 100 100 100
1280 100 100 100 100
1518 100 100 100 100
Packet Size (Bytes)
Spirent third party performance report for Nexus 6004 BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
29
2240 100 100 100 100
4096 100 100 100 100
9100 100 100 100 100
Nexus 6004 - 384 x 10G Unicast 100% Load Latency 1.4 1.2
Average Latency1 (microseconds) 0.8 0.6 0.4 0.2 0 RFC 2544 Unicast L2 Port Pair RFC 2544 Unicast L3 Port Pair
64
80
96
128
256
512
1000
1024
1280
1518
2240
4096
9100
1.028
0.99
1.005
1.023
1.121
1.118
1.111
1.116
1.11
1.107
1.11
1.108
1.101
1.032
1.001
1.011
1.03
1.122
1.121
1.116
1.118
1.113
1.111
1.112
1.11
1.106
Packet Size (Bytes) BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
30
Nexus 6004 - 384 x 10G Multicast 100% Load Latency 1.4 1.2 1
Average Latency (microseconds)
0.8 0.6 0.4 0.2 0
RFC 3918 Mcast L2 One-to-Many Accumulated RFC 3918 Mcast L3 One-to-Many Accumulated RFC 3918 Mcast L2 One-to-Many Distributed RFC 3918 Mcast L2 One-to-Many Distributed
BRKARC-3452
64
80
96
128
256
512
1000 1024 1280 1518 2240 4096 9100
1.004 0.989 1.029 1.017 1.095 1.086 1.079 1.084 1.083 1.082 1.082 1.08 1.077 1.067 1.051 1.061 1.064 1.087 1.083 1.079 1.081 1.08 1.079 1.078 1.076 1.073 0.951 0.949 0.954 0.962 1.019 1.042 1.043 1.042 1.042 1.042 1.043 1.043 1.043 0.96 0.958 0.963 0.974 1.019 1.044 1.044 1.044 1.044 1.044 1.044 1.044 1.044
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
31
Nexus 6004 - 384 x 10G Unicast and Multicast 100% Load Jitter 12 10 8
Average Jitter (nanoseconds)
6 4 2 0
Unicast L2 Jitter (ns) Unicast L3 Jitter (ns) Multicast L2 Jitter (ns) Multicast L3 Jitter (ns)
64 8 8 9 9
80 5 5 6 5
96 8 9 9 9
128 5 5 5 5
256 6 6 5 5
512 8 8 8 8
1000 6 6 6 6
1024 7 7 7 7
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
32
1280 6 6 6 6
1518 10 10 10 9
2240 10 10 9 9
4096 9 9 9 9
9100 9 10 9 9
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
33
Cisco Nexus 5672 Internal Architecture Switch Fabric Unified Fabric Controller (UFC)
Supervisor UPC 1
UPC 2
UPC 3
UPC 4
UPC 5
UPC 6
UPC 0
FC PHY
48xSFP+
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
6xQSFP+
Cisco Public
34
CPU
Cisco Nexus 56128 Internal Architecture Slot 2
Slot 1
2x QSFP+
24xSFP+ FC PHY
FC PHY
UPC 7
UPC 8
UPC 9
2x QSFP+
24xSFP+ FC PHY
FC PHY
UPC 10
UPC 11
UPC 12
Supervisor Unified Fabric Controller (UFC)
UPC 1
UPC 2
UPC 3
UPC 4
48xSFP+ BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
35
UPC 5
UPC 6
2x QSFP+
2x QSFP+
CPU
Cisco Nexus 6004 Internal Architecture Switch Fabric UFC 1
UPC 1
UFC 2
UPC 2
UPC 3
UFC 3
UFC 4
UPC 4
UPC 1
UPC 2
UPC 0
FC PHY
20xSFP+
12xQSFP+
CPU Supervisor
12x40G LEM (M12Q) BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
20x10G LEM (M20UP) Cisco Public
36
Unified Port Controller - UPC ASIC • Multimode MAC; built-in PHY for 1, 10, and 40GE
• Packet parsing and rewriting • Lookup engine and access control: L2, L3, FabricPath, ACL, FCoE, and policing • Buffering and queuing: Buffer management, PFC for lossless traffic, queuing (Strict Priority Queuing and DWRR), and packet replication (SPAN and multicast) • Extra fabric bandwidth for SPAN and multi-destination traffic
Front Panel Ports Twelve 10 GE
Fabric Connections 224Gbps UPC 448Gbps
224Gbps
Three 40 GE BRKARC-3452
UPC
© 2014 Cisco and/or its affiliates. All rights reserved.
448Gbps Cisco Public
37
Cisco Nexus 5600/6000 Packet Processing Flow Ingress Pipeline
VLAN Table MAC Table ARP/ND Table IPv4/IPv6 Routing Table FabricPath FC routing Table
Ingress UPC Buffer Allocation Buffer Accounting PAUSE Signal
1, 10, and 40 GE
Unicast VOQ (8 per egress port)
Forwarding Lookup MAC
Packet Rewrite
Parser Ingress ACL Ingress Policing
Buffer Management
Buffer BRKARC-3452
Security ACL QoS ACL SPAN Filter Policing/Stats Storm Control
Ingress SPAN Replication
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
38
8000 Multicast VOQs
Unified Crossbar Fabric
Nexus 5600/6000 Packet Processing Flow Egress Pipeline
Unified Crossbar Fabric
Egress UPC Unicast buffer Multicast congestion management
Unicast Queue
Buffer Management
Scheduling
Multicast Replication
Egress ACL Egress Policing*
Editing
TX SPAN Replication
Buffer Multicast Queue
To Ingress Buffer Management
*Egress policing is not supported with current software BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
39
MAC
Cisco Nexus 5600/6000 Key Forwarding Tables Host table: 256,000 -entry hashing table; actual capacity is slightly less than 256,000
Host table default carving: 128,000 MAC, 128,000 IP hosts
Host table: Shared between MAC, ARP, and ND and /32 host route
LPM table: 32,000 entries. Also known as summary routes
256,000 Entries
Host Table*
MAC Region
LPM Table(32,000)
Summary Routes
IP Host Region (ARP/ND/Host route/(*,G)) * Hardware table size. Please check configuration limit for software scaling BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
40
Mroute Table(64,000)*
(S,G)
Cisco Nexus 5600/6000 Host Table Carving • In Layer 2, virtualized environments, it can be interesting to have more than 128k MAC addresses on the switch • MAC/ARP Hardware Resource Carving allows to change the size of MAC and IP regions to allocate more space for the MAC region.
256,000 Entries
Host Table
MAC Region
IP Host Region (ARP & ND & /32 Host route)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
41
Cisco Nexus 5600/6000 Carving Profiles • Use of pre-defined carving profiles. • HRT = Host Route Table = IP host region. • STM = Station Table Management = MAC region. Template Profile
Description
hrt-128-stm-128
HRT size: 128k, STM size: 128k (default profile)
hrt-96-stm-160
HRT size: 96k, STM size: 160k
hrt-64-stm-192
HRT size: 64k, STM size: 192k
hrt-32-stm-224
HRT size: 32k, STM size: 224k
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
42
Cisco Nexus 5600/6000 MAC/ARP Resource Carving CLI • Specify the resource template to use: •
•
switch(config)# hardware profile route resource servicetemplate template-name Save the config and reload the switch
• Show commands: • • • •
BRKARC-3452
show show show show
hardware profile route resource hardware profile route resource running-config hardware profile startup-config hardware profile
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
43
template template default route resource template route resource template
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
44
QSFP 40G Bidirectional Transceiver 40G deployment with 10G cabling • QSFP pluggable 40Gbit/s Bidirectional (BiDi) Transceiver that works with existing Multi-mode fiber infrastructure – Using the existing 10Gbit/s Multi-mode infrastructure – QSFP-40G-SR-BD has dual LC Connector – 40Gbit/s BiDi transceiver has two 20Gbit/s channels each transmitted and received simultaneously on two wavelengths
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
45
QSFP-40G-SR-BD Supports 100m with OM3 with MMF and 150m with OM4 (OM4+) MMF 40G-BD
40G-BD Multimode LC Duplex
LC duplex connectors Wavelength: ~850nm and ~900nm Pull-Tab Color: grey
Optical DeMux
40G-BD
10G-SR 10G-SR 10G-SR 10G-SR
Multimode LC Duplex BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
46
QSFP-40G-SR4
40G-SR4
40G-SR4 MPO-12 MPO-12
• Support 100M with OM3 MMF and 150M with OM4 MMF 10G-SR 10G-SR 10G-SR 10G-SR
40G-SR4
MPO-12 LC Breakout BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
47
QSFP-40G-CSR4 • Supports 300M with OM3 MMF and 400M with OM4 MMF 40G-CSR4
40G-SR4/CSR4
• Compatible with 10G SFP-SR
MPO-12 MPO-12
10G-SR 10G-SR 10G-SR 10G-SR
40G-CSR4
MPO-12 LC Breakout BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
• Compatible with Cisco QSFP40G-SR4
48
Multimode Ribbon Fiber 40GE
For QSFP-40G-SR4 and QSFP-40G-CSR4 12-Fiber MPO Connector • MPO/MTP connector with 12 pins
• Use 4 fiber pairs − 4 TX and 4 RX allows for 40GE − 4 unused fibers in the center
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
49
QSFP-40GE-LR4 • QSFP-40GE-LR4 can only be connected to each other. 10KM with Single mode fiber. 4x ROSA
Multiplexer LC
TIA
Host Card
IEEE Standard 40GBASE-LR4
4x TOSA LC
Can’t connect to 10Gbase-LR
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
50
FET-40G • • • •
Low-cost QSFP optical transceiver connecting FEX to Cisco Nexus 6004 Supported on Cisco Nexus 5600/6004 and Nexus 2248PQ-10G Interoperable with FET-10G Support for 100m distance with OM3 Cisco Nexus 5600/6000
Cisco Nexus 5600/6000 FET-40G
FET-40G
FET-10G FET-40G Cisco Nexus 2248PQ-10G BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
51
Cisco Nexus 2232PP and 2232TM-E 2232TM Cisco Nexus 2248TP-E 2248TP
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
52
ACL Types and Features • Security ACL − MAC, IPv4, and IPv6 ACLs − PACL: ACL enabled under L2 interface − VACL: ACL enabled for L2 VLAN traffic − RACL: ACL enabled for routed traffic − RBACL: Role-based ACL for CTS (roadmap feature)
• ACL for QoS classification and Policing • Policy Based Routing (PBR) – User-configured ACL to redirect traffic
• ACL for SPAN/ERSPAN • ACL for control traffic – To redirect control traffic to CPU, not user-configurable – CoPP BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
53
ACL Scaling and TCAM Partition • 4096 Access Control Entries per UPC are organized in blocks of 64 ACE entries. • Each IPv6 ACL (without port range) requires two TCAM entries. Default TCAM Partition
VACL(1024) 4096 Entries
IFACL(1152) QoS(448) RBACL(1024) SPAN(64) Control Traffic(256) BRKARC-3452
switch# sh platform afm info tcam 0 region ifacl ifacl tcam TCAM configuration for asic id 0: [ vacl tcam]: range 0 - 1023 [ifacl tcam]: range 1024 - 2175 * [ qos tcam]: range 2176 - 2623 [rbacl tcam]: range 2624 - 3775 [ span tcam]: range 3776 - 3839 [ sup tcam]: range 3840 - 4095
TCAM [ifacl tcam]: [v:1, size:1152, start:1024 end:2175] TCAM Region In use tcam entries: 24 Usage 1024-1039,2168-2175
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
54
TCAM Carving • TCAM Carving allows to change the size of each region • Create a template: switch(config)# hardware profile tcam resource template t1 switch(config-tcam-templ)#
• Set region sizes: switch(config-tcam-templ)# ifacl ? <320-3584> Enter size of ifacl increments)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
55
region
(in
64
entry
TCAM Carving • Similarly, other region sizes can be changed: switch(config-tcam-templ)# ? ifacl Configure size of ifacl region no Change size to default value qos Configure size of qos region rbacl Configure size of rbacl region span Configure size of span region vacl Configure size of vacl region
• Display the configuration for a template along with the current usage: switch(config)# show hardware profile tcam resource template name
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
56
TCAM Carving switch(config-tcam-templ)# show hardware profile tcam resource template name t1 ------------------------------------------------------------------Template name: t1 Current state: Created Region Size-allocated Current-size Current-usage Available/free ------------------------------------------------------------------Vacl 1152 1024 28 996 Ifacl 1024 1152 266 971 Rbacl 1152 1152 3 1149 Qos 448 448 30 418 Span 64 64 6 58 Sup 256 256 74 182 -------------------------------------------------------------------
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
57
TCAM Carving • Commit a template: switch(config)# hardware profile tcam resource service-template t1 Details of the t1 template you are trying to commit are as follows: ------------------------------------------------------------------------------Template name: t1 Current state: Committed (Startup/Running) Region
Features
Size-allocated
Current-size
Current-usage
Available/free
------------------------------------------------------------------------------Vacl
Vacl
1152
2048
11
2037
[…] ------------------------------------------------------------------------------
Template committed. Please do the following for the template to be applied: 1> Save running config :
"copy running-config startup-config"
2> Reboot the switch
"reload”
BRKARC-3452
:
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
58
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
59
Why Analytics? – A Data Center Today: • 1G -> 10G -> 40G -> 100G
• Virtualized • More and more applications • Those applications are more and more complex (Hadoop…)
• More and more flows • Latency is very important • Ports speed mismatch • Buffer discrepancy between the devices • With spine/leaf, design is less complex, but there’s more speed and more applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
60
Switched Port Analyzer (SPAN) Switch
Host-B
Host-A
Monitor Destination
Monitor Source
Sniffer Device
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
61
Encapsulated Remote SPAN (ERSPAN) Host-B
GRE Encapsulated monitored traffic
IP Cloud Host-A
Monitor Destination
Monitor Source
Sniffer Device BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
62
Cisco Nexus 5600/6000 SPAN SPAN Features
Nexus 5600/6000
Nexus 5500
Total SPAN sessions
31 bi-directional (16 with current NXOS)
4 bi-directional
Local SPAN sessions
31 bi-directional (16 with current NXOS)
4 bi-directional
ERSPAN sessions Prioritize data over SPAN
16 Yes (through scheduling)
4 Yes (SPAN policing)
Line-rate SPAN throughput
Yes
No
ERSPAN destination session ERSPAN with 1588 PTP timestamp
Yes Yes*
No No
MTU Truncated SPAN and ERSPAN ACL filter for SPAN and ERSPAN
Yes Yes
Yes Yes
SPAN-on-Drop SPAN-on-Latency
Yes Yes
No No
SPAN with multiple destination ports
Yes (each destination port uses one SPAN session)
No
*Software roadmap feature BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
63
High-Performance SPAN Extra Fabric Bandwidth for SPAN
Scheduler Prioritizes Production Data
Data
Unified Crossbar Fabric
SPAN Separate Buffer Pool for SPAN
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
64
SPAN on Drop - NX-OS 7.0(1)N1(1), Q1/2014
SPAN-On-Drop
Tail-Drop
SPAN-on-Drop
BRKARC-3452
Ingress Data Buffer
Monitoring Station
SPAN Buffer
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
65
SPAN on Drop - NX-OS 7.0(1)N1(1), Q1/2014 SPAN-on-Drop • Copies the dropped packet to capture device • Correlate the packet drop with application • Works for unicast packets only • Packet drops can be monitored on ingress only • There is a dedicated buffer space for SPAN, so SPAN-on-Drop does not affect production traffic • Supports both local SPAN and ERSPAN • Configuration uses existing SPAN commands • One SPAN-on-Drop session is supported • Can have multiple source ports, and multiple destination ports • Source cannot be FEX HIF port. But FEX fabric port is supported • Source port(s) can be a part of a SPAN-on-Drop session, and a local SPAN session simultaneously BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
66
SPAN on Drop - NX-OS 7.0(1)N1(1), Q1/2014
SPAN-on-Drop
The source interface is the ingress port for which we want to monitor drops.
switch(config)# monitor session 1 type span-on-drop
switch(config-span-on-drop)# source interface e3/1 switch(config-span-on-drop)# destination interface e3/2
switch(config)# monitor session 2 type span-on-drop-erspan switch(config-span-on-drop-erspan)# source interface e3/1 switch(config-span-on-drop-erspan)# destination ip 100.1.1.2
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
67
Latency Monitoring Latency Monitoring: Min: 856ns Max: 1208ns Avg: 901ns
T0 T1 – T0
T0 T1 – T0
• •
TimeStamp Packet
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Latency Histogram: Latency Range: 800ns - 10usec Stats: Within the Range: x packets Out of the Range: y packets
68
FIFO Latency Per Port-Pair Data
Latency LatencyMonitoring Monitoring - NX-OS 7.0(1)N1(1), Q1/2014 • Latency Monitoring provides {min, average, max} latency between a specified port pair and also maintains latency histogram (accuracy in few nanosecs) • By default instantaneous Latency Monitoring is enabled between pair of ports • Latency Histogram can be enabled for specific port-pair to provide histogram instead of instantaneous mode • Measures switch latency for each packet, no sampling required • Fully implemented in HW, no CPU impact, no traffic impact
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
69
Instantaneous Mode • Enabled by default on all pairs of ports. No configuration is required switch# show hardware profile latency monitor interface e1/23 interface e1/22 -------------------------------------------------------------------------------Latency Statistics in nano seconds -------------------------------------------------------------------------------| Ingress Port | Egress Port | Minimum | Maximum | Average | -------------------------------------------------------------------------------| Ethernet1/22| Ethernet1/23| 856| 1208| 901| --------------------------------------------------------------------------------
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
70
Custom Histogram • Count the number of packets falling in a specific range of latency • Example for port-pair e1/1 and e1/3: switch(config)# interface e1/3 switch(config-if)# packet latency low-latency 800 high-latency 10000 mode custom interface e1/1
switch# show hardware profile latency monitor histogram interface e1/3 interface e1/1 -------------------------------------------------------------------------------| Egress Interface : Ethernet1/3 Ingress Interface : Ethernet1/1 | -------------------------------------------------------------------------------| All Latency Values are in nano seconds | -------------------------------------------------------------------------------| Range | 800 <= Latency < 10000 | Outside the first range | ------------------------------------------------------------------------------| counter | 3542818572 | 16290988 |
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
71
SPAN-on-Latency Latency Monitoring: Min: 856ns Max: 1208ns Avg: 901ns
T0 T1 – T0
T2 T3 – T2
TimeStamp Packet
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
If Latency Threshold > 10 usec: SPAN to 1/64
Monitoring Station
72
SPAN-on-Latency • This feature replicates packets from the interface when latency exceeds a configured threshold • Latency threshold is per-port • This again helps to identify the victim of congestion • Replicated traffic uses the SPAN buffer so it doesn't impact the production traffic • One SPAN-on-Latency session is supported in hardware • Uses similar SPAN CLI, with new session type “span-on-latency” • ERSPAN-on-latency is also supported BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
73
SPAN-on-Latency Configuration • The SPAN-on-Latency source port is the egress port on which we monitor the latency • The SPAN-on-Latency session makes a copy of all high-latency packets egressing on this port, coming from any ingress port • Sample configuration to SPAN packets if latency goes beyond 10us when egressing on e1/3: Always Tx: packets egressing on 1/3 (any source) with latency >10us will be replicated to the SPAN dest 1/4
monitor session 1 type span-on-latency source interface Ethernet1/3 tx destination interface Ethernet1/4 interface Ethernet1/3 hardware profile latency monitor threshold 10001 interface Ethernet1/4 switchport mode monitor BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
74
SPAN-on-Latency • Source port can be an regular ethernet port, not a port-channel. Can be a port-channel member
• Source port cannot be FEX HIF port. But FEX fabric port is supported • Multiple sources can be configured – latency threshold is per SPAN-ondrop TX source port
• A SPAN-on-Latency source port cannot be in another SPAN session • Destination is only a single ethernet port, not port-channel • Only one destination port can be configured
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
75
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
76
Cisco Nexus 5600/6000 Multicast Features Summary • IGMP snooping for IGMPv1, v2, and v3 • PIM-SM • PIM-SSM without vPC and PIM-SSM with vPC+ • PIM-BiDir without vPC and PIM-BiDir with vPC+ • PIM Policy • MSDP • Anycast RP with PIM (RFC 4610) • Anycast RP with MSDP • 8000 IGMP snooping entries
• 8000 mroutes with vPC, 16000 mroutes without vPC supported by software BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
77
Efficient Multicast Replication • Optimized multicast replication • Fabric replication and egress replication; one copy is replicated to egress UPC, where there is a receiver—minimizing the traffic load on the switch fabric and eliminating the switch fabric congestion
• Line-rate multicast replication Egress UPC
Ingress UPC
BRKARC-3452
Unified Crossbar Fabric
© 2014 Cisco and/or its affiliates. All rights reserved.
Egress UPC
Cisco Public
78
Multicast VOQ • 8000 multicast VOQs to eliminate HOLB and help ensure high throughput • Tracks the fan-out of the egress UPC; packets with different egress UPC fanouts are assigned to different VOQs so that it can be scheduled and served independently Ingress UPC
Multicast VOQ Unicast VOQ
Egress UPC 1 224.1.1.1
Unified Crossbar Fabric
Egress UPC 2
224.1.1.1
224.1.1.2
Egress UPC 3 224.1.1.2
224.1.1.2 BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
79
Cisco Nexus 5600/6000 Key Forwarding Tables Host table: 256,000 -entry hashing table; actual capacity is slightly less than 256,000
Host table default carving: 128,000 MAC, 128,000 IP hosts
Host table: Shared between MAC, ARP, and ND and /32 host route
LPM table: 32,000 entries. Also known as summary routes
256,000 Entries
Host Table*
MAC Region
• Mroute table: 64,000 entries
LPM Table(32,000)
Summary Routes
IP Host Region (ARP/ND/Host route/(*,G)) * Hardware table size. Please check configuration limit for software scaling BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
80
Mroute Table(64,000)*
(S,G)
MET Table • 64,000 entries in MET table • Currently supported: 8,000 vPC, 16,000 non-vPC
Ingress UPC
Multicast VOQ Unicast VOQ
• Today the MET table is the limiting factor for multicast scale MET: 123
4
16384 16385
122 123
MET: 123
1
1, 4, 5
5
65536
Egress UPC 1 2, 10
2 MET: 123
16384 16385 10
Switch Fabric BRKARC-3452
122 123
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
65536 81
Egress UPC 2
Multicast Hashing over Port Channel • N5600/6000 implements flow based hashing for multi-destination traffic, and it supports multicast load sharing over Port Channel with 5-tuple packet header.
• Traffic is replicated to all egress UPCs where Port Channel member resides • Egress UPC runs hash calculation and one egress port is chosen to send out multicast packets. The UPC ASIC that is not supposed to send out packet will drop packet (Egress UPC 1 in the example)
Multicast VOQ
Ingress UPC
Unicast VOQ
Egress UPC 1
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Port 3 Selected
Unified Crossbar Fabric
Port 1 Port 2
Egress UPC 2 Port 3 Port 3 Selected Port 4
Cisco Public
82
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
83
Cisco Nexus 5600/6000 QoS Features • Eight classes of service; – 2 reserved for control traffic, 6 for data traffic
• Traffic classification – DSCP, CoS, and ACL
• Strict Priority Queuing and DWRR – DCBX 802.1Qaz
• Packet marking – DSCP, CoS, and ECN
• Ingress and egress policing – 4096 policers per ASIC
• No drop system class • Flexible buffer management BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
84
Cisco Nexus 5600/6000 QoS Processing Flow Ingress UPC Trust CoS/DSCP L2/L3/L4 info with ACL MAC
Traffic Classification
Ingress Policing
MTU Checking
Per-class Buffer Usage Monitoring
VOQs for Unicast (8 per egress port)
Multicast Queues
Truncate or Drop Packets if MTU is Violated
Egress Queues
PAUSE ON/OFF Signal MAC
Unicast
ECN Marking
Egress Policing
Egress Scheduling Multicast
Egress UPC BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Strict Priority + DWRR Scheduling 85
Crossbar Fabric
Ingress Cos/DSCP Marking
If Buffer Usage Crosses Threshold: • Tail drop for drop class • Assert pause signal to MAC for no-drop system class
Increased Packet Buffer • 25MB packet buffer is shared by every three 40GE ports or twelve 10GE ports. 150MB of total buffer
• Nexus 56128:
300MB of total buffer
• Nexus 6004:
800MB of total buffer
Ingress UPC
Multicast VOQ
16MB
Unicast VOQ
• Nexus 5672:
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Egress UPC
9MB Unified Crossbar Fabric
Cisco Public
86
Ingress Buffer - Flexible Buffer Management • Shared buffer is good for burst absorption.
UPC Ingress Buffer (16MB)
• Dedicated buffer is good for predictable performance for each port.
SPAN
• N5600/6000 buffer management is flexible: dedicated plus shared.
Shared Packet Buffer ~14MB
• Long-distance FCoE, video editing (i.e., AVID), Big Data, and distributed storage
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Control
Port 1 Dedicated
87
Port 2 Dedicated
Port 3 Dedicated
Tune Shared vs Dedicated Buffer Allocation at Ingress • “queue-limit” under “network-qos” policy specifies the dedicated buffer for each port and each class. • Without “queue-limit” each class of service will get 100 KB of dedicated buffer. • The size of dedicated buffer can be different for different classes of service. The policy applies to all ports in the chassis. • Total ingress buffer minus the dedicated buffer and buffer for control and SPAN will be in the shared buffer pool.
• The following example sets the dedicated buffer for “class-default” to be 400 KB for all ports. switch(config)# policy-map type network-qos Policy-buffer switch(config-pmap-nq)# class type network-qos class-default switch(config-pmap-nq-c)# queue-limit 400000 bytes switch(config-pmap-nq-c)# system qos switch(config-sys-qos)# service-policy type network-qos Policy-buffer BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
88
Egress Buffer Design • 9-MB buffer is reserved for egress traffic
UPC Egress Buffer (9MB)
• Unicast traffic can be buffered at egress and ingress.
Unicast Buffer
• Multicast is buffered at egress in case of interface oversubscription.
Dedicated per port
Multicast Buffer Shared ~6MB
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
89
Unicast Flow Control and Buffering • Unicast is queued at egress buffer pool first and then ingress buffer after egress queue is full. • Take advantage of ingress buffer from multiple port or ASIC for unicast burst absorption. • Ensure fairness among multiple ingress ports
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
90
Identify Packet Drops Due to Congestion switch# show interface ethernet 1/7
switch# show queuing interface ethernet 1/7 Ethernet1/1 queuing information: TX Queuing qos-group sched-type oper-bandwidth 0 WRR 100
Ethernet1/1 is down (SFP not inserted) Dedicated Interface
RX 0 input error 0 short frame 0 overrun 0 underrun 0 ignored 0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop 0 input with dribble 0 Rx pause
TX
RX Queuing qos-group 0
q-size: 100160, HW MTU: 1500 (1500 configured) drop-type: drop, xon: 0, xoff: 0 Statistics: Pkts received over the port : 710025080 Ucast pkts sent to the cross-bar : 474177592 Mcast pkts sent to the cross-bar :0 Ucast pkts received from the cross-bar : 0 Pkts sent to the port :0 Pkts discarded on ingress :
235847488 input discard Unicast drop at ingress
0 unicast packets 0 multicast packets 0 broadcast packets 0 output packets 0 bytes 0 jumbo packets 0 output error 0 collision 0 deferred 0 late collision
235847488
Per-priority-pause status (Inactive), Tx (Inactive)
0 lost carrier 0 no carrier 0 babble 0 output discard 0 Tx pause Broadcast/Multicast/Unknown
Ingress drop for each class
unicast flooding drop at egress BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Dedicated buffer for this class
91
: Rx
Identify Congested Egress Port with VOQ Counters I X I A I X I A
1/7
Ingress UPC 3
Unified Crossbar Fabric Egress UPC 3
1/8 Ingress UPC 4
1/10
Using “show interface” CLI, we noticed drops on ingress ports How do we know which egress port is affected? BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
92
I X I A
Step 1: Map Interface to UPC Number N6004# show hardware internal bigsur all-ports Bigsur Port Info: Port |asic|inst|inst| name |idx |slot|asic|eport|logi|flag|adm|opr|if_index|diag|ucVer ---------+----+----+----+-----+----+----+---+---+--------+----+----sup1 |0 |17 |0 | 0 - |0 |b3 |en |dn |15010000|pass| 0.00 sup0 |0 |17 |0 | 1 - |1 |b3 |en |dn |15020000|pass| 0.00 40gb1/1 |1 |0 |0 | 0 - |0 |b3 |en |dn |1a070000|pass| 0.00 40gb1/2 |1 |0 |0 | 1 - |1 |b3 |en |dn |1a071000|pass| 0.00 40gb1/3 |1 |0 |0 | 2 - |2 |b3 |en |dn |1a072000|pass| 0.00 40gb1/4 |2 |0 |1 | 0 - |3 |b3 |en |dn |1a073000|pass| 0.00 40gb1/5 |2 |0 |1 | 1 - |4 |b3 |dis|dn |1a074000|pass| 0.00 40gb1/6 |2 |0 |1 | 2 - |5 |b3 |dis|dn |1a075000|pass| 0.00 40gb1/7 |3 |0 |2 | 0 p |6 |b3 |en |up |1a076000|pass| 0.00 40gb1/8 |3 |0 |2 | 1 p |7 |b3 |en |up |1a077000|pass| 0.00 40gb1/9 |3 |0 |2 | 2 p |8 |b3 |en |up |1a078000|pass| 0.00 40gb1/10 |4 |0 |3 | 0 p |9 |b3 |en |up |1a079000|pass| 0.00 40gb1/11 |4 |0 |3 | 1 - |10 |b3 |dis|dn |1a07a000|pass| 0.00 40gb1/12 |4 |0 |3 | 2 p |11 |b3 |en |up |1a07b000|pass| 0.00 Done. BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
93
Step 2: Identify Congested Egress Port with VOQ Counters N6004# show platform software qd info counters voq asic-num 3 +----------+------------------------------+------------------------+-----------+ | port| TRANSMIT| TAIL DROP| HEAD DROP| +----------+------------------------------+------------------------+-----------+ Eth1/8 QUEUE-3 3888954496 2248133231 0 +----------+------------------------------+------------------------+-----------+
Congested egress port and class of service N6004# show platform software qd info counters voq asic-num 4 +----------+------------------------------+------------------------+-----------+ | port| TRANSMIT| TAIL DROP| HEAD DROP| +----------+------------------------------+------------------------+-----------+ Eth1/8 QUEUE-3 2574819434 2248180087 0 +----------+------------------------------+------------------------+-----------+
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
94
Ingress UPC number
Nexus 5600/6000 Architecture - Agenda Overview Fabric Extender Performance Internal Architecture Transceivers/Cables Access Control List Analytics Multicast Quality of Service Applications BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
95
Nexus 5600/6000 Applications Compact Aggregation Nexus 7000
CORE L3
Nexus 7000 Nexus 6004
AGG. vPC to Nexus 7000 or 6004
Nexus 6004/ Nexus 5600
ACCESS
Nexus 2000
FEX
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
96
Nexus 5600/6000 Applications Large Scale Fabric (Layer 2 or Layer 3)
CORE
Nexus 7000
L3 SPINE
Nexus 6004
FabricPath Nexus 6004/ Nexus 5600+2000
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
LEAF
Cisco Public
97
Nexus 5600/6000 Applications High Performance Computing (HPC)
CORE/AGG.
Nexus 6004
L3/L2 Fabric Nexus 3548 Nexus 5600/6004
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
ACCESS
Cisco Public
98
Target Q3 CY14
Nexus 6004X Chassis • Nexus 6004X has 8 slots (similar to the 6004EF chassis) • Nexus 6004X chassis supports VXLAN functionality
• Provides Flexible Ordering Options – mix/match 10/40G LEMs • Supports new 12 x 40G LEM VXLAN capable (backward compatible with Nexus 6004EF chassis) • VXLAN capable LEM will work in non-VXLAN mode in 6004EF chassis • Same software as on Nexus 6004 • Supports all the optics, FEX models as on Nexus 6004
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
99
Nexus 5600/6000 Key Takeaways Rich Feature set
STP/vPC/FabricPath/DFA, FEX, L3, VXLAN, Adapter-FEX/VM-FEX
Performance
Line rate L2 and L3 with low latency
Scalability
256K MAC / 128K ARP / 32K LPM routes, 16K bridge domains with segment ID
Innovations
Intelligent SPAN, Buffer monitoring, Latency monitoring
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
100
Nexus 2000/5600/6000 More Information • http://cisco.com/go/nexus2000 • http://cisco.com/go/nexus5000 • http://cisco.com/go/nexus6000
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
101
Complete Your Online Session Evaluation • Give us your feedback and you could win fabulous prizes. Winners announced daily. • Complete your session evaluation through the Cisco Live mobile app or visit one of the interactive kiosks located throughout the convention center. Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
102
Continue Your Education • Demos in the Cisco Campus • Walk-in Self-Paced Labs • Table Topics
• Meet the Engineer 1:1 meetings
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
103