The quoted ports are the ‘stand alone’ non clustered port names (not node1′s port names once clustered). In a SRX cluster the PIM slots on node1 start at the last PIM slot of node0 + 1. For example, a SRX240 cluster’s node1 starts at PIM 5. It’s control link port is effectively ge-5/0/1).

 *fab0 and fab1 interfaces (Data Link) are always configurable, e.g.:

• set interfaces fab0 fabric-options member-interfaces ge-0/0/2
• set interfaces fab1 fabric-options member-interfaces ge-5/0/2

Here is the conf:

set system archival configuration transfer-on-commit
set system archival configuration archive-sites "scp://junos@x.x.x.x/data/configs/DEVICE" password "bla"
set system archival configuration archive-sites "scp://junos@y.y.y.y/data/configs/DEVICE" password "bla"

More info: http://www.juniper.net/techpubs/en_US/junos9.5/information-products/topic-collections/swconfig-system-basics/junos-software-system-management-router-configuration-archiving.html

Configuration (all centrally from the Director)

To build a new Fabric you need to

Create aliases for nodes

• set fabric aliases node-device SERIAL ALIAS_NAME

Node Replacement

Replacing a node, and keeping the configuration is EXTREMELY easy due to the ‘replace pattern’ feature of Junos.

• Repatch cables
• replace pattern OLD_SERIAL with NEW_SERIAL
• commit

I would like to thanks Juniper APAC and Yeu Kuang Bin for this excellent opportunity and very knowledgable training.

What is QFabric?

The most simple explanation of QFabric I can explain is that it is basically a Juniper EX Virtual Chassis on steroids. The internal workings of the switch have been broken apart to be MUCH MORE scalable and Juniper have insured that there are no single points of failure, only selling the design with fully redundant components.

The QFabric components are:

• Director Group – 2 x QFX3100 (Control Plane)

• Interconnects – 2 x QFX3008-I (Backplane / Fabric)
• 2 REs per Interconnect

• Nodes (Data Plane)
• Server Groups – 1 – 2 per group

40GE DAC cable (1m,3m,5m lengths)
40GB – QSFP+ (quad small form-factor pluggable plus) – 40 gig uses MTP connector

QFabric Node Discovery

Control Plane

The control plane is discovered automatically, it depends on being configured with a pre-defined Juniper configuration in order to discover the nodes via a pre-defined method when you turn the QFX3500 into fabric mode.

Data/Fabric Plane

The fabric plan is what makes QFabric as scalable as it is. Once again a predefined HA design is supplied and the directors perform the following tasks:

1. Discovers, builds & Maintains Topology of the Fabric
2. Assembles the entire topology
3. Propagates path information to all entities

Please note: The information presented here is from my own point of view. It is no way associated with the firm beliefs of Juniper Networks (TM) or ICT Networks (TM).

The goal, as network engineers is to use all bandwidth when it’s available (if feasible) and avoid legacy protocols to stop layer 2 loops such as Spanning-Tree. vPC from Cisco and VC technologies allow LACP (Link Control Aggregation Protocol) links to span physical chassis, allow the network engineer to avoid single points of failure and harness all available bandwidth. If a physical chassis was lost, you would still be operation in a degraded fashion, e.g. 1/2 the available bandwidth until the second chassis returned.

To configure the Cisco Nexus side you would require the following configuration on each vPC configured chassis. I found that VLAN pruning can be happily done and a Natvie VLAN1 is not needed if CDP is not mandatory (I did not test making CDP able to traverse the trunk through the Juniper – Would love to hear if someone does!).

conf t

interface port-channel69
  description Good practice
  switchport mode trunk
  vpc 69
  mtu 9216
  switchport trunk allowed vlan 69

interface Ethernetx/x
  channel-group 69 mode active

Handy Cisco Debug Commands:

• show vpc
• show run interface port-channel69 member
• show vpc consistency-parameters int port-channel 69
• show port-channel summary

The Juniper side would only require the following, this configuration is identical (you just choose different member interfaces) even if you don’t have a Virtual Chassis configuration.

set interfaces xe-0/0/39 ether-options 802.3ad ae0
set interfaces xe-1/0/39 ether-options 802.3ad ae0
set interfaces ae0 description "Good Practice"
set interfaces ae0 mtu 9216
set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members pr0nNet

set vlans pr0nNet vlan-id 69
set vlans pr0nNet l3-interface vlan.69 #If a L3 RVI is required

Handy Juniper Debug Commands:

• show interface terse ae0
• show lacp interfaces (you want your interfaces to be collecting and distributing)
• show interface ae0 extensive

Please let me know if I have done anything that is not optimal – always eager to learn, I am definitely not (and proud of it) a Cisco expert.

UPDATE (20120401): Seems Juniper has addressed and fixed this bug …
More info:
http://kb.juniper.net/InfoCenter/index?page=content&id=KB21713&actp=RSS 

It seems you can not have the ‘IP Spoofing’ screen enabled when sending IPSec Dynamic VPN traffic ingressing into the zone with the screen set. This traffic is dropped by the screen which can be seen via a ‘security flow traceoption flag basic-datapath’:

• ‘packet dropped, drop by spoofing check.’

So removing (or deactivating) the ip spoofing check solved the problem:

• deactivate security screen ids-option from-Internet ip spoofing

Kind of lame, the spoofing screen sounds a good idea on your Internet facing interfaces, but seems a no no if you want dynamic VPNs. That is all. Hopefully eventually Juniper make this check smarter.

set groups member0 system host-name member0
set groups member1 system host-name member1
set groups member2 system host-name member2
set apply-groups member0
set apply-groups member1
set apply-groups member2

Download Script

Sample Output:

cooper@dfbit:~/scripts/iperf-parse$ ./iperf-summary.py
------------------------------------
   --   IPERF CSV Summariser   --
-- Cooper Lees <me@cooperlees.com --
------------------------------------
-- SUMMARY --
- 20111212103043 to 20120103090052
- 1004 runs of IPERF
- Averages:
-	Average Sent			= 2.64M
-	Average Received		= 2.28M
-	Average Send Bandwidth		= 985.75K
-	Average Receive Bandwidth	= 805.12K
-	Max Send Bandwidth		= 1.08M (at 20111230183021)
-	Max Receive Bandwidth		= 837.16K (at 20120102113052)
------------------------------------

Code:

#!/usr/bin/python

# date,sender-ip,sender-port,receiver-ip,receiver-port,id,interval,transfer,bandwidth
# 20111212103043,10.120.15.8,45020,10.120.13.120,5001,5,0.0-21.4,2490368,931080
# 20111212103109,10.120.15.8,5001,10.120.13.120,57022,4,0.0-24.2,2228224,736145

FILENAME = 'client-iperf.log'

RUNS = 1

MAX_BANDWIDTH_SENT = 0
MAX_BANDWIDTH_SENT_DATE = 0
MAX_BANDWIDTH_RECEIVED = 0
MAX_BANDWIDTH_RECEIVED_DATE = 0

TOTAL_BANDWIDTH_SENT = 0
TOTAL_BANDWIDTH_RECEIVED = 0

TOTAL_SENT = 0
TOTAL_RECEIVED = 0

def convert_bytes(bytes):
	bytes = float(bytes)
	if bytes >= 1099511627776:
		terabytes = bytes / 1099511627776
		size = '%.2fT' % terabytes
	elif bytes >= 1073741824:
		gigabytes = bytes / 1073741824
		size = '%.2fG' % gigabytes
	elif bytes >= 1048576:
		megabytes = bytes / 1048576
		size = '%.2fM' % megabytes
	elif bytes >= 1024:
		kilobytes = bytes / 1024
		size = '%.2fK' % kilobytes
	else:
		size = '%.2fb' % bytes
	return size

f = open(FILENAME)
l1 = f.readline().strip().split(',')
l2 = f.readline().strip().split(',')
while l2 and l2[0] != '':
	if RUNS == 1:
		START = l1[0]

	BW_SENT = int(l1[8])
	BW_RECEIVED = int(l2[8])

	TOTAL_SENT = TOTAL_SENT + int(l1[7])
	TOTAL_RECEIVED = TOTAL_RECEIVED + int(l2[7])

	TOTAL_BANDWIDTH_SENT = TOTAL_BANDWIDTH_SENT + int(l1[8])
	TOTAL_BANDWIDTH_RECEIVED = TOTAL_BANDWIDTH_RECEIVED + int(l2[8])

	if BW_SENT > MAX_BANDWIDTH_SENT:
		MAX_BANDWIDTH_SENT = BW_SENT
		MAX_BANDWIDTH_SENT_DATE = l1[0]

	if BW_RECEIVED > MAX_BANDWIDTH_RECEIVED:
		MAX_BANDWIDTH_RECEIVED = BW_RECEIVED
		MAX_BANDWIDTH_RECEIVED_DATE = l2[0]

	END = l2[0]
	RUNS = RUNS + 1
	l1 = f.readline().strip().split(',')
	l2 = f.readline().strip().split(',')

f.close()

print "------------------------------------"
print "   --   IPERF CSV Summariser   --   "
print "-- Cooper Lees <me@cooperlees.com --"
print "------------------------------------"
print "-- SUMMARY --"
print "- %s to %s" % ( START, END )
print "- %d runs of IPERF" % RUNS
print "- Averages:"
print "-\tAverage Sent\t\t\t= %s" % convert_bytes((TOTAL_SENT / RUNS))
print "-\tAverage Received\t\t= %s" % convert_bytes((TOTAL_RECEIVED / RUNS))
print "-\tAverage Send Bandwidth\t\t= %s" % convert_bytes((TOTAL_BANDWIDTH_SENT / RUNS))
print "-\tAverage Receive Bandwidth\t= %s" % convert_bytes((TOTAL_BANDWIDTH_RECEIVED / RUNS))
print "-\tMax Send Bandwidth\t\t= %s (at %s)" % (convert_bytes((MAX_BANDWIDTH_SENT)), MAX_BANDWIDTH_SENT_DATE)
print "-\tMax Receive Bandwidth\t\t= %s (at %s)" % (convert_bytes((MAX_BANDWIDTH_RECEIVED)), MAX_BANDWIDTH_RECEIVED_DATE)
print "------------------------------------"

Cron Job Script to Collect Data:

#!/bin/bash

SERVER="x.x.x.x"
LOG="client-iperf.log"
TIME="5"

echo "--> Starting iperf client @ $(date) ..." | tee -a $LOG

if [ "$1" == "-v" ]; then
        iperf -t $TIME -c $SERVER -r -y C | tee -a $LOG
else
        iperf -t $TIME -c $SERVER -r -y C >> $LOG
fi

echo "--> Finished iperf client @ $(date)" | tee -a $LOG