QFabric Part 1 – Explained and Explored First Hand

NOTE: Interconnects DO NOT interconnect to each other

Node aliasing allows administrators to give nodes a meaningful name and is used when talking about specific interfaces for specific nodes or node groups

• Id the nodes via beaconing (the LCD screen) or serial number on chassis.
• e.g. set fabric aliases node-device P6969-C NODE-0
• This name is used to reference ports and assign the node to a group (discussed next)

Node groups are used to allow the infrastructure to be divided up and allow the director to know what type of cofiguration to push to a nodes routing-engine. The local routing engine still performs some tasks, predominately to allow scale. A group can contain a maximum of 2 nodes. A group with 2 nodes is know as a redundant server group (It is a 2 node virtual chassis under the covers). Due to this, a redundant server group can have multi-chassis ae (aggregated ethernet) interfaces. There is one other type of group known as the Network node group. This group looks after all routing and l2 loop information, such as OSPF and spanning tree. All vlan routing etc. is done by these nodes.

1. Network Node Group (1 per QFabric - Max 8 nodes)
2. Server Group (Redundant Server Group optional - 2 nodes)
1. Qfabric automatically creates a redundant server group if two nodes exist in a server group (via a form of virtual chassis).

Now cause each node has an 'alias' (discussed above) to reference a port in configuration you now use:

• NODE_ALIAS:INT_TYPE-x/x/x.x
• e.g. NODE-0:xe-0/0/1.0

Aggregated interfaces can be deployed - Across chassis in a redundant server group or on one chassis in a server group:

• GROUP_NAME:ae0.0
• e.g. RACK-42-1:ae0.0

QFabric can also function with port in FC and FCoE mode. There are some limitations to this feature today, but can provide an excellent mechanism to create redundant paths back through the Fabric to the SAN FC based network. This will be discussed in a dedicated post in my QFabric series.

QFabric, for a Data Center is ready today and works extremely well. It can allow a HUGE number of 10gb (and soon to be 40gb) ports to allow huge data movement around a DC at low latency. It is also effectively one single point of management for all your nodes, unless something goes wrong of course. For a campus, with end users, QFabric does not have many key features that we use today either with the MX or EX range. It could be used for large campuses as the aggregation or core (especially when more IPv4 and IPv6 routing is supported) and feed 10gb out to EX switches to provide the 'edge'. The coming 'micro' fabric is also interesting, which will allow for a more compelling footprint within a smaller data center.

• Single switch in regards to management and functionalty
• No TRILL or other L2 bridging redundancy protocols required
• Ultra redundant design - Enforced by Juniper
• No half way deployment, people can't go in half assed !
• The simple well thought out HA deployment/design - Common install = easier to debug for JTAC / Engineers like myself
• Scalability - Can see how big DCs could benefit from having 1 gigantic switch
• Road map looks good - Key features and hardware are coming

• AFL (Advanced Feature License) required for IPv6 (when it arrives)
• PLEASE Juniper - Can we have IPv6 for free or I will never get customers to deploy it
• This really frustrates me ... You may be able to tell 🙂
• Limitation of 1 unit per interface
• No vlan tagging and multiple units in Network Groups
• Can work around by turning port into trunk and assigning multiple L3 interfaces
• The need for legacy SAN infrastructure in order to use FC/FCoE (discussed in part 3)
• No ability to have a full 48 Copper SFP 1gb interfaces in a node for legacy non 10gig equipment
• The QFX3500 can not fit physically the SFPs in top and bottom rows
• This could be handy to keep legacy equipment and as it's replaced change the SFP to a 10g SFP+

• The Micro Fabric - will allow more use cases
• Full SNMP interface statistics for all nodes through the director
• Currently testing this with Zenoss in the Juniper Lab - Has not worked so far
• The ability to ensure node's RE's and PSU etc. are also a plus (have not tested / read the MIBs yet - so could be possible)
• Be able to downgrade and system wide request system rollback from the director
• Full Q-in-Q Support
• Fully self contained FC/FCoE support

Part 2 - Deploying and Configuring

Part 3 - FCoE and SAN with QFabric

Part 4 - QFabric eratta (possibly - not sure yet ...)