I was lucky enough to be one of the first APAC partner engineers to get my hands on Juniper's new QFabric gigantic scalable switch technology. I have even beat some of Juniper's own SEs. In general, it rocks, but does have some features and fine tuning, this will come. This post is an introduction to QFabric, with my likes, dislikes and feature wish-list.
I would like to thanks Juniper APAC and Yeu Kuang Bin for this excellent opportunity and very knowledgable training.
What is QFabric?
The most simple explanation of QFabric I can explain is that it is basically a Juniper EX Virtual Chassis on steroids. The internal workings of the switch have been broken apart to be MUCH MORE scalable and Juniper have insured that there are no single points of failure, only selling the design with fully redundant components.
The QFabric components are:
- Director Group - 2 x QFX3100 (Control Plane)
- Interconnects - 2 x QFX3008-I (Backplane / Fabric)
- 2 REs per Interconnect
- Nodes (Data Plane)
- Server Groups - 1 - 2 per group
40GE DAC cable (1m,3m,5m lengths)
40GB - QSFP+ (quad small form-factor pluggable plus) - 40 gig uses MTP connector
QFabric Node Discovery
The control plane is discovered automatically, it depends on being configured with a pre-defined Juniper configuration in order to discover the nodes via a pre-defined method when you turn the QFX3500 into fabric mode.
The fabric plan is what makes QFabric as scalable as it is. Once again a predefined HA design is supplied and the directors perform the following tasks:
- Discovers, builds & Maintains Topology of the Fabric
- Assembles the entire topology
- Propagates path information to all entities
- Id the nodes via beaconing (the LCD screen) or serial number on chassis.
- e.g. set fabric aliases node-device P6969-C NODE-0
- This name is used to reference ports and assign the node to a group (discussed next)
- Network Node Group (1 per QFabric - Max 8 nodes)
- Server Group (Redundant Server Group optional - 2 nodes)
- Qfabric automatically creates a redundant server group if two nodes exist in a server group (via a form of virtual chassis).
- e.g. NODE-0:xe-0/0/1.0
Aggregated interfaces can be deployed - Across chassis in a redundant server group or on one chassis in a server group:
- e.g. RACK-42-1:ae0.0
- Single switch in regards to management and functionalty
- No TRILL or other L2 bridging redundancy protocols required
- Ultra redundant design - Enforced by Juniper
- No half way deployment, people can't go in half assed !
- The simple well thought out HA deployment/design - Common install = easier to debug for JTAC / Engineers like myself
- Scalability - Can see how big DCs could benefit from having 1 gigantic switch
- Road map looks good - Key features and hardware are coming
- AFL (Advanced Feature License) required for IPv6 (when it arrives)
- PLEASE Juniper - Can we have IPv6 for free or I will never get customers to deploy it
- This really frustrates me ... You may be able to tell 🙂
- Limitation of 1 unit per interface
- No vlan tagging and multiple units in Network Groups
- Can work around by turning port into trunk and assigning multiple L3 interfaces
- The need for legacy SAN infrastructure in order to use FC/FCoE (discussed in part 3)
- No ability to have a full 48 Copper SFP 1gb interfaces in a node for legacy non 10gig equipment
- The QFX3500 can not fit physically the SFPs in top and bottom rows
- This could be handy to keep legacy equipment and as it's replaced change the SFP to a 10g SFP+
- The Micro Fabric - will allow more use cases
- Full SNMP interface statistics for all nodes through the director
- Currently testing this with Zenoss in the Juniper Lab - Has not worked so far
- The ability to ensure node's RE's and PSU etc. are also a plus (have not tested / read the MIBs yet - so could be possible)
- Be able to downgrade and system wide request system rollback from the director
- Full Q-in-Q Support
- Fully self contained FC/FCoE support
Please note: The information presented here is from my own point of view. It is no way associated with the firm beliefs of Juniper Networks (TM) or ICT Networks (TM).
This Post Has 3 Comments
Like the article. Would love to speak with you
Happy to chat mate. Fire me an email (firstname.lastname@example.org) and we can chat there or try and organise a Skype call etc.
How many servers are you planning on deploying and how many virtual machines on each? I am trying to figure out how the 8,000 entry ARP cache is going to work when I have more than 8,000 MAC-to-IP bindings in the ARP table – seems like a major screw up but have not had the opportunity t play with it…