Tag Archives: cisco

Get your Fabric Interconnects talking!

This post will walk thru configuring Fabric Interconnects. Read this after you have racked & cabled Fabric Interconnects. Before you begin, make sure you have the following available:

  • 3x IP Addresses (used for each FI and 1 VIP)
  • Cable L1,L2 and Mgmt ports on each FI
  • DNS Address(s)

We will walk thru initial configuration via the GUI web interface of the FI & console config of the secondary FI.

Connect to the console port of your primary FI and open your console session to begin. If you happen to connect to the console before you apply power, you’ll have the opportunity to watch the FI boot. If you watch closely you’ll notice an NXOS underpinning, it will look something like this.

Version 2.00.1201. Copyright (C) 2009 American Megatrends, Inc.
Booting kickstart image: bootflash:/installables/switch/ucs-6100-k9-kickstart.5.0.3.N2.4.02b.bin...................................................................................
.............................................................Image verification
 OKUsage: init 0123456SsQqAaBbCcUu
INIT: [   10.975503] I2C - Mezz present
Starting system POST.....
  Executing Mod 1 1 SEEPROM Test:...done (0 seconds)
  Executing Mod 1 1 GigE Port Test:....done (32 seconds)
  Executing Mod 1 1 PCIE Test:.................done (0 seconds)
  Mod 1 1 Post Completed Successfully
POST is completed
can't create lock file /var/lock/mtab~208: No such file or directory (use -n flag to override)
S10mount-ramfs.supnuovaca Mounting /isan 3000m
Mounted /isan
Creating /callhome..
Mounting /callhome..
Creating /callhome done.
Callhome spool file system init done.
nohup: redirecting stderr to stdout
autoneg unmodified, ignoring
autoneg unmodified, ignoring
Checking all filesystems..... done.
Checking NVRAM block device ... done
The startup-config won't be used until the next reboot.
.
Loading system software
Starting the smart check..
Uncompressing system image: bootflash:/installables/switch/ucs-6100-k9-system.5.0.3.N2.4.02b.bin



27+1 records in
27+1 records out
20480 bytes (20 kB) copied, 8.9105e-05 s, 230 MB/s
ethernet end-host mode on CA
FC end-host mode on CA
n_port virtualizer mode.
---------------------------------------------------------------
INIT: Entering runlevel: 3
touch: cannot touch `/var/lock/subsys/netfs': No such file or directory
Mounting other filesystems:  mount: /dev/hd-usbslot1 is not a valid block device
[FAILED]
touch: cannot touch `/var/lock/s
/isan/bin/muxif_config: fex vlan id: -f,4042
Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Added VLAN with VID == 4042 to IF -:muxif:-

Changing of vsh_perm
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
---------------------
enabled fc feature
---------------------
System is coming up ... Please wait ...
2019 Sep 20 15:52:51  %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files begin  - clis
2019 Sep 20 15:52:53  %$ VDC-1 %$ Sep 20 15:52:53 %KERN-0-SYSTEM_MSG: [   10.975503] I2C - Mezz present  - kernel
System is coming up ... Please wait ...
System is coming up ... Please wait ...
2019 Sep 20 15:53:01  %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files end  - clis
2019 Sep 20 15:53:01  %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: init begin  - clis
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
2019 Sep 20 15:54:31  %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 1 has come online
System is coming up ... Please wait ...
nohup: appending output to `nohup.out'
2019 Sep 20 15:54:53 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: Running in PIO stats mode  - carmelusd

If you smash enter a few times on the console screen you’ll get this.


           ---- Basic System Configuration Dialog ----

  This setup utility will guide you through the basic configuration of
  the system. Only minimal configuration including IP connectivity to
  the Fabric interconnect and its clustering mode is performed through these steps.

  Type Ctrl-C at any time to abort configuration and reboot system.
  To back track or make modifications to already entered values,
  complete input till end of section and answer no when prompted
  to apply configuration.


   Switch can now be configured from GUI. Use https://192.168.45.66 and click
   on 'Express Setup' link. If you want to cancel the configuration from GUI and go back,
   press the 'ctrl+c' key and choose 'X'. Press any other key to see the installation progress from GUI

I plan to show both methods of the configuration, the GUI method for the primary FI and the console for the subordinate.

If you take note of the code from the snippet above, you’ll see an IP address. You will need to set your local IP to an address within that same subnet, then connect the computer to the MGMT port on the FI, once that is done you can browse to the FI GUI.

I set my local IP to 192.168.45.50

Now, connect to the FI GUI and you should get something like this

Open the URL and continue past any security alerts, the next page will take you to the beginning of the ‘Express Setup’


After you’ve clicked Express Setup, you will be presented with a very minimal configuration screen. This is where the IP Addresses mentioned earlier will come in handy.

I will be continuing with a clustered configuration. Select Fabric A as this is your first FI in the cluster. Then fill in the remaining information:

  • Virtual Address (address of the cluster)
  • System Name
  • Admin password
  • Mgmt IP Address (The individual IP of THIS FI)
  • Mgmt IP Netmask
  • Gateway
  • DNS Server
  • Domain Name (optional)
  • UCS Central IP & Shared Secret (optional)

Once all of the details have been filled in, click SUBMIT and your config will begin writing to the FI, you will be presented with the following:

If you switch back to the console you’ll see this on the console:

You can now login with the credentials of admin : (the password you just set)

AWESOME, HALF OF THE CONFIG IS DONE!! The second half we will complete via the console, and you will only need 1 piece of information.

  • Password of the primary FI
  • IP Address of the secondary FI (the one you are about to config)

Move your console cable from the primary FI, to the secondary and smash enter a few times. Your console should read something like this:

Type 'reboot' to abort configuration and reboot system or Type 'X' to cancel GUI configuration and go back to console  or Press any other key to see the installation progress from GUI (reboot/X) ?

Type X and hit Enter, next you will be asked to configure via GUI or Console, type Console and hit Enter.

Enter the configuration method. (console/gui) ? console

This is where you should have already connected your L1,L2 cables, if done correctly the FI will detect it’s peer and prompt you to enter the password of the primary FI.

Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the cluster. Continue (y/n) ? y

  Enter the admin password of the peer Fabric interconnect:
    Connecting to peer Fabric interconnect... done
    Retrieving config from peer Fabric interconnect... done
    Peer Fabric interconnect Mgmt0 IPv4 Address: X.X.X.X
    Peer Fabric interconnect Mgmt0 IPv4 Netmask: 255.255.255.0
    Cluster IPv4 address          : X.X.X.X

    Peer FI is IPv4 Cluster enabled. Please Provide Local Fabric Interconnect Mgmt0 IPv4 Address

  Physical Switch Mgmt0 IP address : X.X.X.X

Hit Enter and you will be asked to save the config, type Yes to save the config

Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes
  Applying configuration. Please wait.

Fri Sep 20 16:09:40 UTC 2019
  Configuration file - Ok


Cisco UCS 6200 Series Fabric Interconnect
HX-Production-B login:

Thats all! You have now configured BOTH FIs! If this made no sense to you, please find more information on Cisco’s UCS Platform and their Fabric Interconnects using the link below. In simple terms, the Fabric Interconnects allow you to virtualize your hardware. If that sounds cool you should also click the link to learn more.

https://www.cisco.com/c/en/us/products/servers-unified-computing/index.html