I found that jool has very good tutorials, but all the commands to get going are hidden in these large tutorials. Here are the steps I took to get it working on Ubuntu 20.04 on both a Raspberry Pi + Protectli Vault.

Please pre-read and refer to to Jool's Documentation for more information.

I have two Ubuntu 20.04 routers at home that run jool. Both routers/firewalls use NFTables so I'm just using jool in netfilter mode. When direct nftables support is implemented, I will move to this setup.

Quick Start/Setup jool

  • On ubuntu 20.04 it's just an apt install
    • apt install jool-dkms jool-tool
  • sudo modprobe jool
    • Add jool to /etc/modules to make persistent
  • Add Stateful NAT64 pool
    • jool instance add --netfilter --pool6 64:ff9b::/96
    • Here I used a oneshot systemd service to add on boot
[Unit]
Description=Add NAT64 netfilter pool6 to jool

[Service]
Type=oneshot
ExecStart=/usr/bin/jool instance add --netfilter --pool6 64:ff9b::/96

[Install]
WantedBy=multi-user.target

Handy Commands

  • See instace
    • jool instance display
    • jool instance status
  • See sessions
    • jool session display
  • Global config
    • jool global display
  • Overall stats
    • jool stats display

Testing

Try and Ping + traceroute to Google's main IPv4 NS anycast address 8.8.8.8 via IPv6 64:ff9b::8.8.8.8:

                              My traceroute  [v0.93]
coopbuntu (fd00:1::10)                                   2020-12-14T04:37:45+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                         Packets               Pings
 Host                                  Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. fd00:1::2                           0.0%    11    2.5   2.3   2.1   2.6   0.1
 2. (waiting for reply)
 3. 64:ff9b::6022:7b58                  0.0%    11    9.9  13.6   9.9  20.7   3.6
 4. 64:ff9b::6022:7576                  0.0%    11   18.6  14.0  11.3  18.6   2.7
 5. 64:ff9b::6022:7932                  0.0%    10   14.0  17.1  13.2  21.9   3.0
 6. 64:ff9b::6022:b5                    0.0%    10   20.6  22.3  18.1  28.2   3.5
 7. 64:ff9b::6022:301                   0.0%    10   22.1  18.9  17.0  22.1   1.5
 8. 64:ff9b::4a7d:3072                  0.0%    10   26.2  21.3  17.7  36.2   5.9
 9. 64:ff9b::6caa:e636                  0.0%    10   24.1  19.2  17.3  24.1   2.5
10. 64:ff9b::4a7d:fc97                  0.0%    10   19.6  21.1  18.0  27.3   3.1
11. 64:ff9b::808:808                    0.0%    10   17.1  18.2  16.5  24.5   2.4

Once ICMP works, move on to tcp.

  • ssh -v 64:ff9b::173.255.255.199

Session Table

  • jool session display is your friend to see current translations
    • --numeric stops the non parallel DNS resolution
Every 1.0s: sudo jool session display                                  home1.cooperlees.com: Mon Dec 14 04:42:17 2020

---------------------------------
(ESTABLISHED) Expires in 1:59:31.440
Remote: us.cooperlees.com#ssh   fd00:1::10#48656
Local: 66.214.99.163#61019      64:ff9b::adff:ffc7#22
---------------------------------
(V4_FIN_V6_FIN_RCV) Expires in 0:03:14.796
Remote: 5.85.222.35.bc.googleusercontent.com#http       fd00:1::10#43868
Local: 66.214.99.163#62581      64:ff9b::23de:5505#80
---------------------------------

Mr Aijay Adams and I am back making my Fireplace Internet / Smart device controllable. Now, via a very sexy Web UI, when I'm heading back to Chateau Tahoe, I can turn my fireplace on to be ready as soon as I walk in the door. Sexy warmth controlled by a sexy custom made API.

  • A goal was to keep the original switch working too, so we can be Cave people as well!

Web UI

Gorgeous Web 0.69 design.

Install Photos

Tech Specs

Hardware:

  • Raspberry Pi 4
    • Relay Hat on the GPIO

Software:

Firestarter API:

  • / - Status of the fireplace
  • /turn_off - Turn the fireplace off
  • /turn_on - Turn thr fireplace off

Are you using the latest Linux kernel firewall?. Here are some notes I've saved that I use and forget all the time.
I plan to add to this as I do more. Hopefully it helps you work something out one day.

Note: I am using inet tables combining my IPv4 and IPv6 rulesets.

List Tables

sudo nft list table inet filter -n -a
sudo nft list table inet nat -n -a

  • -n: numeric
  • -a: handle (object handles)

Add a rule

nft insert rule inet filter OUTPUT position 0 icmpv6 type {nd-router-advert} drop

Delete a rule

nft delete rule inet filter OUTPUT handle 41

ICMPv6 Types

Noting some handy IPv6 ICMP types. I use nftables to block RAs when my WAN is down.

  • nd-router-advert == 134

tcpdump expressions

  • tcpdump -v -i en0 'ip6[40] = 134'

Recently in the Terragraph project I work on we changed from RPM to OPKG to removes some dependencies (e.g. perl) and make our overall image size smaller. I've never driven OPKG, but know RPM, so I made this cheat sheet for my shit memory.

I'm cheap so I don't have a Table plugin - So used Python to generate me one ūü§†

+--------------------------+-------------------------------+
| RPM Cmd                  | OPKG Cmd                      |
+--------------------------+-------------------------------+
| rpm -qa                  | opkg list-installed           |
| rpm -qf <FILE>           | opkg search <FILE>            |
| rpm -i[vh] --force <PKG> | opkg install [--force*] <PKG> |
| rpm -e --force <PKG>     | opkg remove [--force*] <PKG>  |
+--------------------------+-------------------------------+
- opkg Force Examples:
  --force-depends|--force-downgrade|--force-remove

I often use a lot of PyPI CLI tools. Here is an example of how to get them easily installed and kept up to date via Ansible on Ubuntu >= 18.04.

Install base pip via apt then run pip:

- name: Get Python3 pip
  package:
    name: python3-pip
    state: latest

- name: Add some handy Python PyPI Tools
  pip:
    name: "{{ item }}"
    extra_args: --upgrade
  with_items:
    - "black"
    - "coverage"
    - "mypy"
    - "pip"
    - "setuptools"

Enjoy up to date core Python tools + handy CLIs for dev work.

Please do NOT use on a Production host ...

Recently a teammate and I have come across a frame forwarding issue with ECMP on a hardware ASIC in a device I work on where the use of Flow labels are used in the ECMP hash. This was interesting as we found iperf was not setting the Flow label at all, unless you specify the -L option and due to this we saw TCP traffic taking different paths, contradictory to what we thought we had configured in our FIB and what we actually wanted.

This sparked interest in me then wondering how popular platforms set the IPv6 Flow label for the different protocols; that being, ICMPv6, TCP and UDP. The Flow label being at Layer 3, I would expect it used the same for each protocol, but I could not find literature to back this theory up. So I fired up Wireshark on Mac, Linux and Windows to find out what they do. Here are my results I found.

If you want to know more about what Flow Labels are I would reccomened the following links:

  • Wikipedia:¬†https://en.wikipedia.org/wiki/IPv6_packet#Fixed_header
  • RFC:¬†https://tools.ietf.org/html/rfc6437

Summary

With each protocol the client and the server maintained consistent Flow labels for the 'session' as expected, except for Windows with ICMPv6 Requests! Here Windows set the Flow label to 0 (0x00000000).

Tests Performed

To get my results I ran:

  • ping6 -c 2 us.cooperlees.com
    (ICMPv6)
    - ping -6 us.cooperlees.com on Windows
  • ssh -6 us.cooperlees.com
    (TCP)
    - Used putty on Windows
  • Raw NTP UDP Query
    Python 3 Code: https://pastebin.com/RDBRqG0G
    (UDP)

Linux

Test Distro: Ubuntu 18.04
Test Kernel: 4.15.0-23-generic

ICMPv6
- Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
- Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
- Different Flow label for sender and receiver for each UDP packet as expected.

Mac OS X

Test Version: 10.13.6 17G65
Test Kernel: Darwin Kernel Version 17.7.0

ICMPv6
- Different Flow label, but consistent for the 2 ping packets on each ICMPv6 Type 128/129 packet from sender and receiver

TCP
- Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
- Different Flow label for sender and receiver for each UDP packet as expected.

Windows

Test Version: Microsoft Windows [Version 10.0.16299.371]

ICMPv6
- Windows sets the ICMPv6 Type 128 (request) IPv6 Flow label to 0x00000000!
(I also noticed different DSCP for traffic class)

TCP
- Different Flow label for sender and receiver but consistent across the SSH connection.

UDP
- Different Flow label for sender and receiver for each UDP packet as expected.

It took me far to long to find out how to config this, so I'm sharing it to be more searching on the Internets.

Scenario:

- NXOS BGP 'upsteam' / 'north bound' peer sharing the default route only
- Linux box runs Quagga sharing it's /56 routable behind it

Nexus Conf

feature bgp

router bgp 65170
  router-id 69.69.69.70
  address-family ipv6 unicast
    network 0::/0
  neighbor 69::69 remote-as 65169
    address-family ipv6 unicast

 

Quagga Conf

bgp.conf:

router bgp 65169
 bgp router-id 69.69.69.69

 neighbor 69::1 remote-as 65170

 address-family ipv6 unicast
  network 2400:0000:0000:6900::/56

 neighbor 69::1 activate

log file /var/log/bgpd.log
log stdout

From time to time I get asked (and I even have to ask a coworker) for the best way to install a Python modules (especially ones with entry points) into a virtualenv¬†and still edit / develop with them. It seems ‚Äčpip install's '-e' is very unknown.

pip install -e /path/you/are/editing

Will allow you to develop and run with all dependencies install in your virtualenv.

Happy dev'in!

 

Ever have to update/merge a PR on BitBucket with Mercurial? I couldn't find documentation anywhere, so doing so here:

  1. hg up BOOKMARK_NAME
  2. hg merge [--preview] -r REV
  3. If EDITOR is not set:
    export EDITOR=vim
  4. hg resolve --all
  5. hg commit -m "Merge with default"
  6. hg push --allow-anon

Especially because:

https://github.com/python/cpython/commit/47320a652e872003f3dd3a9db4243067b09dd316#diff-c6a3fa0ad7b17f8e32f340835a4e5353

ūüôā