IPv6 + Flow labels

Recently a teammate and I have come across a frame forwarding issue with ECMP on a hardware ASIC in a device I work on where the use of Flow labels are used in the ECMP hash. This was interesting as we found iperf was not setting the Flow label at all, unless you specify the -L option and due to this we saw TCP traffic taking different paths, contradictory to what we thought we had configured in our FIB and what we actually wanted.

This sparked interest in me then wondering how popular platforms set the IPv6 Flow label for the different protocols; that being, ICMPv6, TCP and UDP. The Flow label being at Layer 3, I would expect it used the same for each protocol, but I could not find literature to back this theory up. So I fired up Wireshark on Mac, Linux and Windows to find out what they do. Here are my results I found.

If you want to know more about what Flow Labels are I would reccomened the following links:

  • Wikipedia: https://en.wikipedia.org/wiki/IPv6_packet#Fixed_header
  • RFC: https://tools.ietf.org/html/rfc6437

Summary

With each protocol the client and the server maintained consistent Flow labels for the ‘session’ as expected, except for Windows with ICMPv6 Requests! Here Windows set the Flow label to 0 (0x00000000).

Tests Performed

To get my results I ran:

  • ping6 -c 2 us.cooperlees.com
    (ICMPv6)
    – ping -6 us.cooperlees.com on Windows
  • ssh -6 us.cooperlees.com
    (TCP)
    – Used putty on Windows
  • Raw NTP UDP Query
    Python 3 Code: https://pastebin.com/RDBRqG0G
    (UDP)

Linux

Test Distro: Ubuntu 18.04
Test Kernel: 4.15.0-23-generic

ICMPv6
– Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

Mac OS X

Test Version: 10.13.6 17G65
Test Kernel: Darwin Kernel Version 17.7.0

ICMPv6
– Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

Windows

Test Version: Microsoft Windows [Version 10.0.16299.371]

ICMPv6
– Windows sets the ICMPv6 Type 128 (request) IPv6 Flow label to 0x00000000!
(I also noticed different DSCP for traffic class)

TCP
– Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
– Different Flow label for sender and receiver for each UDP packet as expected.

Mac OS X 10.7 Lion + ATEN UC232A USB to Serial Adapter

32- and 64-bit PL2303 drivers for OS X 10.6 are available here.

You’ll need to modify, as root, the ‘/System/Library/Extensions/ProlificUsbSerial.kext/Contents/Info.plist ‘ file after installing the driver to suit the USB manufacturer and device ID. For the ATEN UC232A, examples below appear to work for it (for me in 10.7).

To obtain the IDs, Wayne Roberts (who informed my via the SAGE-AU mailing lists), used ‘USB Prober.app’ which comes with the developer tools/XCode.

Modify the current lines of the XML:

[text]
# <key> can be the Hex values as ‘Vendor’_’Product’, Wayne thinks this is more cosmetic however.
<dict>
<key>0557_2008</key>
<dict>

—-

# <idProduct> and <idVendor> should be the decimal of the respective values, as per USB Prober.app
<key>idProduct</key>
<integer>8200</integer>
<key>idVendor</key>
<integer>1367</integer>
[/text]

Once you’ve done this, either restart the machine or run ‘kextunload’ and ‘kextload’ on ProlificUsbSerial.kext and it should show up as /dev/tty.usbserial.

** If you have installed the UC232A Drivers, either rm or mv the ‘UC-232AC.kext’ before the reboot to avoid conflicts **

Apple Mac OS X Time Machine over NFS (10.5 or greater)

  1. Create a Sparebundle to copy to your NFS/SMB/AFP volume
    hdiutil create -size $SIZEg -fs HFS+J -type SPARSEBUNDLE -volname “Backup of $MACHINENAME” $MACHINENAME_$MACADDRESS.sparsebundle
  2. Use ifconfig and sub in your $MACHINENAME_$MAC_ADRESS into the sparsebundle filename – Remove the ‘:’ chars
    e.g. 0018b31184dd
    Use hostname to get your machines hostname – Don’t use fqdn
    Size can be adjusted at a later date if required – g = gigabytes
  3. Copy the budle to the nfs volume
    cp -rp $MACHINENAME_$MACADDRESS.sparsebundle /Path/to/nfs
  4. Run the defaults command to turn on timemachine for unsupported volumes support (As normal user – don’t sudo)
    defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1
  5. Now go to your System Prefrences and choose time machine
    Choose your NFS/SMB/AFP volume as the timemachine disc