Juniper is selling QFabric as a bundle. Due to this the install has been templated and will be similar in regards to the control plane and getting the fabric up and ready to be configured for each target environment. Every QFabric bundle today must include Juniper Professional Services. Hopefully in the future I (and other partner engineers) will be seen as smart enough to do a QFabric install without Juniper's assistance. I think I could manage it :). Here is the procedure that you and your friendly Juniper Professional Service engineer will complete to get QFabric up and running.
Building the EX4200 VCs for Control Plane
Juniper have extensive configuration documentation with example configurations for the Control Plane for the QFabric components. Please read Juniper's article "Configuring the Virtual Chassis for the QFabric Switch Control Plane" for instructions on building the control plane infrastructure. I will not go into specifics in this blog post for the control plane.
Initial QFabric Components Deployment
This is my recollection and notes taken during the demonstration and explanation from Juniper's current most experienced QFabric installer in APAC of the basic process of getting your QFabric up and running.
- Check BOM against what's been received and power all equipment and test for hardware issues
- Also ensure directors and interconnects are the same version (should not be a problem yet but as 'newer' builds come old stock might pop up)
- Build and power ex4200 VCs for control plane
- I would recommended to upgrade to the JTAC recommended version of Junos on your 4200s
- Patch Up directors into control plane VCs and boot the desired 'master director'
- Complete the console initialisation and then after ~60 seconds boot the slave and complete it's initial configuration
- Patch directors into correct control ports and boot
- Turn each node into 'fabric mode'
- Patch into each interconnect and boot each node
- The directors will adjust the version of Junos if required on the QFX3500 node
- You now have a functional QFabric and can now beging to alias nodes and add them to network/server groups
Configuration (all centrally from the Director)
To build a new Fabric you need to
Create aliases for nodes
- set fabric aliases node-device SERIAL ALIAS_NAME
Create node groups
Always have 1 network-domain and 1 server group (max 2 nodes per server group making it a redundant server group)
- set fabric resources node-group NW-NG-0 network-domain
- set fabric resources node-group NW-NG-0 node-device ALIAS_NAME_X
- set fabric resources node-group PRON-NG node-device PRON_SW1
Further configuration is 'like' a normal EX style configuration, but using the new interface names, for example:
Interface: NODE_ALIAS:xe-0/0/1.0
Aggregated Interface: NODE_GROUP:ae0.0
Aggregated Interface: NODE_GROUP:ae0.0
Handy Debug Commands
- show fabric administration inventory director-group status all
- See the directors status and who is master
- show fabric administration inventory [terse]
- Shows all the hardware the directors have found and are including in the QFabric
- show chassis fabric connectivity
- Shows the connectivity through the interconnects to each nodes
- show fabric aliases
- See the serial to alias mappings
- show fabric inventory
Checking VLANS, the ethernet-switching table etc. commands are all identical to the Juniper EX Switch family.
Power On Sequence
- ex4200 Control Plane VCs
- QFabric Interconnects
- Director Master
- Election of master is based on uptime. Wait for ~60 to boot secondary director node
- Nodes
- I have not tested this, but I would power the network group first, with the members I would prefer to be the masters of the 'vc' first (remember each group with multiple members is an incarnation of VC - same rules apply)
Extra Functions
Node Replacement
Replacing a node, and keeping the configuration is EXTREMELY easy due to the 'replace pattern' feature of Junos.
- Repatch cables
- replace pattern OLD_SERIAL with NEW_SERIAL
- commit