Category: cisco

IPv6 + Flow labels

Recently a teammate and I have come across a frame forwarding issue with ECMP on a hardware ASIC in a device I work on where the use of Flow labels are used in the ECMP hash. This was interesting as we found iperf was not setting the Flow label at all, unless you specify the -L option and due to this we saw TCP traffic taking different paths, contradictory to what we thought we had configured in our FIB and what we actually wanted.

This sparked interest in me then wondering how popular platforms set the IPv6 Flow label for the different protocols; that being, ICMPv6, TCP and UDP. The Flow label being at Layer 3, I would expect it used the same for each protocol, but I could not find literature to back this theory up. So I fired up Wireshark on Mac, Linux and Windows to find out what they do. Here are my results I found.

If you want to know more about what Flow Labels are I would reccomened the following links:

  • Wikipedia: https://en.wikipedia.org/wiki/IPv6_packet#Fixed_header
  • RFC: https://tools.ietf.org/html/rfc6437

Summary

With each protocol the client and the server maintained consistent Flow labels for the ‘session’ as expected, except for Windows with ICMPv6 Requests! Here Windows set the Flow label to 0 (0x00000000).

Tests Performed

To get my results I ran:

  • ping6 -c 2 us.cooperlees.com
    (ICMPv6)
    – ping -6 us.cooperlees.com on Windows
  • ssh -6 us.cooperlees.com
    (TCP)
    – Used putty on Windows
  • Raw NTP UDP Query
    Python 3 Code: https://pastebin.com/RDBRqG0G
    (UDP)

Linux

Test Distro: Ubuntu 18.04
Test Kernel: 4.15.0-23-generic

ICMPv6
– Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

Mac OS X

Test Version: 10.13.6 17G65
Test Kernel: Darwin Kernel Version 17.7.0

ICMPv6
– Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

Windows

Test Version: Microsoft Windows [Version 10.0.16299.371]

ICMPv6
– Windows sets the ICMPv6 Type 128 (request) IPv6 Flow label to 0x00000000!
(I also noticed different DSCP for traffic class)

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

NXOS Simple IPv6 BGP Peering

It took me far to long to find out how to config this, so I’m sharing it to be more searching on the Internets.

Scenario:

– NXOS BGP ‘upsteam’ / ‘north bound’ peer sharing the default route only
– Linux box runs Quagga sharing it’s /56 routable behind it

Nexus Conf

feature bgp

router bgp 65170
  router-id 69.69.69.70
  address-family ipv6 unicast
    network 0::/0
  neighbor 69::69 remote-as 65169
    address-family ipv6 unicast

 

Quagga Conf

bgp.conf:

router bgp 65169
 bgp router-id 69.69.69.69

 neighbor 69::1 remote-as 65170

 address-family ipv6 unicast
  network 2400:0000:0000:6900::/56

 neighbor 69::1 activate

log file /var/log/bgpd.log
log stdout

IPv6 Tacacs+ Support (tac_plus)

Recently @ Facebook we found that we required IPv6 access to TACACS for auth (AAA) for the majority of our production Network Equipment. Tacacs+ (tac_plus) is an old daemon released by Cisco in the late 90s. It still works (even at our scale) and the config was doing what we required, so it was decided that we should add IPv6 Support to it to move forwards until we no longer require TACACS for authentication, authorization and accounting.

IPv6 has been added in true dirty 90s C code style via pre-processor macros. The source is publicly available via a GitHub Repository.

This version is based off F4.0.4.19 with the following patches (full history can be seen in the Git Repository):

  • Logging modifications
  • PAM Support
  • MD5 support
  • IPv6 (AF_INET6) Socket Listening

Readme.md has most of the information you require to build the software and I have included RPM .spec files (that have been tested on CentOS 6). The specs generate two RPMS with tacacs+6 relying on the tacacs+ rpm to be installed for libraries and man pages.

RPMS Build on CentOS 6.5 x86_64 + SRC rpms avaliable here: https://cooperlees.com/rpms/

Usage Tips:

  • Do not add listen directives into tac_plus.conf so that each daemon can load the same conf file (for consistency)
  • Logging:
    • /var/log/tac_plus.acct and tac_plus6.acct are where accounting information will go (as well as syslog) – Logrotate time …
    • /var/log/tac_plus.log and tac_plus6.log is where default debug logs will go
  • Configure syslog to send the LOG_LOCAL3 somewhere useful (this will get both tac_plus and tac_plus6 log information)
  • Pid Files will live in /var/run/tac_plus.pid.0.0.0.0 and tac_plus6.pid.::
  • The RPM does not /sbin/chkconfig –add or enable, so be sure to enable the version of tac_plus you require.

Tested Support on Vendor Hardware

  • Arista EoS (4.13.3F): need to use ‘ipv6 host name ::1’ as TACACS conf can’t handle raw IPv6 addresses (lame) 
  • Cisco NXOS (6.0(2)U2(4) [build 6.0(2)U2(3.6)]):
    feature tacacs+
    tacacs-server key 7 “c00p3rIstheMan”
    tacacs-server host a:cafe::1
    tacacs-server host b:b00c::2
    aaa group server tacacs+ TACACS
    server a:cafe::1
    server b:b00c::2
    source-interface Vlan2001 (ensure what IP request will come from)
  • Juniper: >= Junos 13.3R2.7 required for IPv6 Tacacs (Tested on MX)

I know it’s old school code but please feel free to submit bug patches / enhancements. This should allow us to keep this beast running until we can deprecate it’s need …

Junos Aggregated Ethernet w/LACP and Cisco Nexus Virtual Port Channel

So when I was googiling around looking for working configurations of Junos (EX in this case) AE working with a Cisco vPC (Virtual Port Channel) I could not find any examples … So I said that I would post one. I will not be covering how to set up a VPC, if you’re interested in that side visit Cisco’s guide here. I will also not discuss how to configure a Juniper Virtual Chassis (more info here). The devices used in this example are 2 x Cisco 7k (running NX-OS 4) and 2 x Juniper EX4500 switches (running Junos 11.4R1) in a Mixed Mode virtual chassis with 2 x ex4200s.

The goal, as network engineers is to use all bandwidth when it’s available (if feasible) and avoid legacy protocols to stop layer 2 loops such as Spanning-Tree. vPC from Cisco and VC technologies allow LACP (Link Control Aggregation Protocol) links to span physical chassis, allow the network engineer to avoid single points of failure and harness all available bandwidth. If a physical chassis was lost, you would still be operation in a degraded fashion, e.g. 1/2 the available bandwidth until the second chassis returned.

To configure the Cisco Nexus side you would require the following configuration on each vPC configured chassis. I found that VLAN pruning can be happily done and a Natvie VLAN1 is not needed if CDP is not mandatory (I did not test making CDP able to traverse the trunk through the Juniper – Would love to hear if someone does!).

[plain]
conf t

interface port-channel69
description Good practice
switchport mode trunk
vpc 69
mtu 9216
switchport trunk allowed vlan 69

interface Ethernetx/x
channel-group 69 mode active
[/plain]

Handy Cisco Debug Commands:

  • show vpc
  • show run interface port-channel69 member
  • show vpc consistency-parameters int port-channel 69
  • show port-channel summary

The Juniper side would only require the following, this configuration is identical (you just choose different member interfaces) even if you don’t have a Virtual Chassis configuration.

[plain]
set interfaces xe-0/0/39 ether-options 802.3ad ae0
set interfaces xe-1/0/39 ether-options 802.3ad ae0
set interfaces ae0 description "Good Practice"
set interfaces ae0 mtu 9216
set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members pr0nNet

set vlans pr0nNet vlan-id 69
set vlans pr0nNet l3-interface vlan.69 #If a L3 RVI is required
[/plain]

Handy Juniper Debug Commands:

  • show interface terse ae0
  • show lacp interfaces (you want your interfaces to be collecting and distributing)
  • show interface ae0 extensive

Please let me know if I have done anything that is not optimal – always eager to learn, I am definitely not (and proud of it) a Cisco expert.

30 Levels of NAT Firewall Lab

So I am a very large geek, and proud of it. It hurt to walk past a cupboard at work knowing there was 30+ Cisco PIX 501 firewalls sitting in there collecting dust. One day it dawned on me, I wonder how crap internet would be sitting behind 30 of those slow ass god awful to use and configure firewalls. So here are the results:

Network Diagram

(Click for larger image)

Sample PIX 501 conf:

[plain]
hostname fwX
password cisco
enable password cisco
domain-name cooperlees.com

ip address inside 10.N.0.1 255.255.255.0
ip address outside 10.N1.0.2 255.255.255.0
interface ethernet0 auto
interface ethernet1 100full

route outside 0 0 10.N1.0.1
nat (inside) 1 10.N.0.0 255.255.255.0
global (outside) 1 interface

access-list outbound permit any any
access-group outbound in interface inside

access-list ping_acl permit icmp any any
access-group ping_acl in interface outside
[/plain]

Video of the Results

Thanks to Jason Leschnik, Anthony Noonan, Kyle Seton and Chris Steven for their assistance.

Cisco Acknowledges AJ2010 Network

Awesome stuff. Was a lot of kit rolled out @ the Cataract Scout Park by Alan, Brett, Colin, Todd and myself – Plus many many others !

Good to see the kit was well used for quality videos like:

More info:
http://blogs.cisco.com/news/comments/cisco_helps_the_22nd_australian_scout_jamboree_reach_new_digital_heights/

Maxing out Gig Ethernet with Sun x4500 Thumpers

Well the time has come where I have finally got some hardware that can max out gig ethernet. I sent 3.4 tb in 9 hours! Thats awesome! GG Cisco 3750 and 2 x Sun x4500 Thumpers running Opensolaris snv_105. Good times – I bet the copper was warm 🙂

root@dumper1:sbin> ./zfsMbufferInitialSend.sh
Starting ZFS send to dumper-tmp.ansto.gov.au @ Fri Mar  6 16:02:47 EST 2009
in @  0.0 kB/s, out @ 34.9 MB/s, 3428 GB total, buffer   0% fullll

summary: 3428 GByte in 8 h 57 min  109 MB/s

real    537m40.533s
user    34m32.281s
sys     326m18.227s
Completed @ Sat Mar  7 01:00:34 EST 2009

If you do use x4500s or have the need to zfs send compile mbuffer today! It rocks – I went from 30mbyte a second with SSH to maxing out Gigabit Ethernet. I will post instructions on everything I did soon.