IPv6 + Flow labels

Recently a teammate and I have come across a frame forwarding issue with ECMP on a hardware ASIC in a device I work on where the use of Flow labels are used in the ECMP hash. This was interesting as we found iperf was not setting the Flow label at all, unless you specify the -L option and due to this we saw TCP traffic taking different paths, contradictory to what we thought we had configured in our FIB and what we actually wanted.

This sparked interest in me then wondering how popular platforms set the IPv6 Flow label for the different protocols; that being, ICMPv6, TCP and UDP. The Flow label being at Layer 3, I would expect it used the same for each protocol, but I could not find literature to back this theory up. So I fired up Wireshark on Mac, Linux and Windows to find out what they do. Here are my results I found.

If you want to know more about what Flow Labels are I would reccomened the following links:

  • Wikipedia: https://en.wikipedia.org/wiki/IPv6_packet#Fixed_header
  • RFC: https://tools.ietf.org/html/rfc6437

Summary

With each protocol the client and the server maintained consistent Flow labels for the ‘session’ as expected, except for Windows with ICMPv6 Requests! Here Windows set the Flow label to 0 (0x00000000).

Tests Performed

To get my results I ran:

  • ping6 -c 2 us.cooperlees.com
    (ICMPv6)
    – ping -6 us.cooperlees.com on Windows
  • ssh -6 us.cooperlees.com
    (TCP)
    – Used putty on Windows
  • Raw NTP UDP Query
    Python 3 Code: https://pastebin.com/RDBRqG0G
    (UDP)

Linux

Test Distro: Ubuntu 18.04
Test Kernel: 4.15.0-23-generic

ICMPv6
– Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

Mac OS X

Test Version: 10.13.6 17G65
Test Kernel: Darwin Kernel Version 17.7.0

ICMPv6
– Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

Windows

Test Version: Microsoft Windows [Version 10.0.16299.371]

ICMPv6
– Windows sets the ICMPv6 Type 128 (request) IPv6 Flow label to 0x00000000!
(I also noticed different DSCP for traffic class)

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

IPv6 Tacacs+ Support (tac_plus)

Recently @ Facebook we found that we required IPv6 access to TACACS for auth (AAA) for the majority of our production Network Equipment. Tacacs+ (tac_plus) is an old daemon released by Cisco in the late 90s. It still works (even at our scale) and the config was doing what we required, so it was decided that we should add IPv6 Support to it to move forwards until we no longer require TACACS for authentication, authorization and accounting.

IPv6 has been added in true dirty 90s C code style via pre-processor macros. The source is publicly available via a GitHub Repository.

This version is based off F4.0.4.19 with the following patches (full history can be seen in the Git Repository):

  • Logging modifications
  • PAM Support
  • MD5 support
  • IPv6 (AF_INET6) Socket Listening

Readme.md has most of the information you require to build the software and I have included RPM .spec files (that have been tested on CentOS 6). The specs generate two RPMS with tacacs+6 relying on the tacacs+ rpm to be installed for libraries and man pages.

RPMS Build on CentOS 6.5 x86_64 + SRC rpms avaliable here: https://cooperlees.com/rpms/

Usage Tips:

  • Do not add listen directives into tac_plus.conf so that each daemon can load the same conf file (for consistency)
  • Logging:
    • /var/log/tac_plus.acct and tac_plus6.acct are where accounting information will go (as well as syslog) Logrotate time …
    • /var/log/tac_plus.log and tac_plus6.log is where default debug logs will go
  • Configure syslog to send the LOG_LOCAL3 somewhere useful (this will get both tac_plus and tac_plus6 log information)
  • Pid Files will live in /var/run/tac_plus.pid.0.0.0.0 and tac_plus6.pid.::
  • The RPM does not /sbin/chkconfig –add or enable, so be sure to enable the version of tac_plus you require.

Tested Support on Vendor Hardware

  • Arista EoS (4.13.3F): need to use ‘ipv6 host name ::1’ as TACACS conf can’t handle raw IPv6 addresses (lame) 
  • Cisco NXOS (6.0(2)U2(4) [build 6.0(2)U2(3.6)]):
    feature tacacs+
    tacacs-server key 7 “c00p3rIstheMan”
    tacacs-server host a:cafe::1
    tacacs-server host b:b00c::2
    aaa group server tacacs+ TACACS
    server a:cafe::1
    server b:b00c::2
    source-interface Vlan2001 (ensure what IP request will come from)
  • Juniper: >= Junos 13.3R2.7 required for IPv6 Tacacs (Tested on MX)

I know it’s old school code but please feel free to submit bug patches / enhancements. This should allow us to keep this beast running until we can deprecate it’s need …

ipadm Script for Openindiana / Solaris 11 Ex

I love ipadm. It rocks. Much needed for Solaris and derivatives. I seem to keep forgetting the dam new ipadm commands tho. So I wrote this script to help me set up new Solaris machines and thought I would share.

[bash]
#!/usr/bin/bash

INTERFACE=$1
ADDRESS=$2
GATEWAY=$3

V6AUTO=1

errorCheck()
{
if [ $? -ne 0 ]; then
echo "ERROR: $@"
exit 69
fi
}

# Handle Args
if [ $# -ne 3 ]; then
echo "ERROR: Invalid arguments"
echo "Usage:nt$0: INTERFACE ADDRESS/CIDR-MASK GATEWAY/DEFAULTROUTER"
exit 1
fi

ipadm create-if $INTERFACE
errorCheck "Unable to create-if $INTERFACE"

ipadm create-addr -T static -a local=${ADDRESS} ${INTERFACE}/v4static
errorCheck "Unable to set static v4 on $INTERFACE"

if [ $V6AUTO -ne 0 ]; then
ipadm create-addr -T addrconf ${INTERFACE}/v6addr
errorCheck "Unable to set v6 autoconf on $INTERFACE"
fi

if [ $GATEWAY != "" ]; then
route add default $GATEWAY
errorCheck "Unable to set default router to $GATEWAY"
else
echo "!–> Not setting gateway as none was set …"
fi

echo "–> Finished setting $ADDRESS on $INTERFACE with $GATEWAY default route …"
[/bash]