Featured post

Getting started with Ansible

First of all, what is Ansible?

Ansible is an open-source automation framework with one of it’s largest backers being IBM/Red Hat. It is offered in 2 flavors.

  1. Ansible Core – Free, no GUI
  2. Ansible Tower – Paid with full support and backing from Red Hat Support

Ansible is gaining more traction when compared with other automation frameworks, such as Salt or Puppet. This is because Ansible is agent-less, is written with a declarative language, it uses simple YAML files lastly it can be used across your entire datacenter stack, meaning you could have a single ‘script’ that could talk to your physical network switch, configure a BMC on your physical server, then deploy a virtual machine in your hypervisor and finally install/configure an application. The same ‘script’ could be broken out into individual tasks and re-used for another project. Ansible has the support of many OEMs and vendors who produce official modules, some of the biggest names include Cisco, Dell, Zabbix, Microsoft. If a module does not already exist for a task you’re trying to accomplish you’re able to write and publish your own module!

Lets jump into a few Ansible key terms you should know going forward.

  • YAML – a human-readable data-serialization language
  • Playbook – are the basis for a really simple configuration management and multi-machine deployment system, can declare configurations, but they can also orchestrate steps of any manual ordered process
  • Inventory – a file that will be used alongside a playbook for targeting machines/devices
  • Ansible Vault – a module that can encrypt any structured data file used by Ansible
  • Play/Task – a declarative piece of code. When you execute the play/task, it’s called a playbook. You can assemble multiple play/tasks into a single file to assemble a more complex playbook.

Ansible really shows its power when you can tie multiple technologies into a playbook to complete a normally complex task. You can also use it to build a ‘standard build’ or ‘standard config’ for various deployments, because it’s declarative you can set how you want the configuration to be, and Ansible is able to take care of the details behind the scenes to ‘set’ your configuration, meaning you don’t need to understand how to get to that config, just that you WANT that config. Below you will see a very simple ‘play’ that will tie most of this together. It’s a small YAML file, that will run as a playbook to install the latest version of Apache using a yum installer.

- name: install the latest version of Apache
    name: httpd
    state: latest

As you progress into the Ansible world, you can tie more and more of these simple tasks to create more complex playbooks. See a more complex playbook below. YAML is a very easy to read language and you likely will be able to just read the code and understand what is happening when you execute the playbook.

- name: This sets up an httpd webserver
  hosts: centos1
  - name: Install apache packages 
      name: httpd
      state: present
  - name: ensure httpd is running
      name: httpd 
      state: started
  - name: Open port 80 for http access
      service: http
      permanent: true
      state: enabled
  - name: Restart the firewalld service to load in the firewall changes
      name: firewalld 
      state: restarted

In future posts we will walk thru installing/configuring and writing our own playbooks using Ansible Core. If you can’t wait until then, feel free to read up on all these things on your own.

Ansible Network Automation – Cisco IOS

This post will give a quick overview of setting up your Ansible deployment to be able to use Ansible IOS network modules, these modules can be put together to build a playbook that can be used in an initial switch deployment as a base config, or could be setup with variables so jr members of your team can be used to make changes to a switch config.

This post assumes that you have already installed ansible core and have successfully pinged a remote device, such as a Linux server.

First thing we will do is setup our group_vars file to tell Ansible what method of communication to use to talk with our new Cisco IOS host. Let’s create a new file under /group_vars, in my machine the folder is located under /etc/ansible/group_vars.

I will create ios.yml under /etc/ansible/group_vars.

 group_vars]$ touch ios.yml

Now that I have created the file, I’ll edit it so that ansible will know to reference back to this file to understand how to communicate with our IOS host we will add shortly to our hosts.ini file. The file should have the following:

ansible_connection: network_cli
ansible_network_os: ios
ansible_user: ## FILL IN USER ##
ansible_ssh_pass: ## FILL IN PASS ##
ansible_become: yes
ansible_become_method: enable
ansible_become_pass: ## FILL IN ENABLE PASS ##

#Note this is not best practice to hard-code credentials in here, but this is just for demonstration purposes. In production you would likely want to use a variable for credentials and/or pair this with ansible-vault, which we will cover later.

Lets run thru a quick breakdown of each of these settings.

ansible_connection : This is the key variable that ansible reads to identify what type of device it will be communicating with. In our case, we will be talking to a Cisco IOS switch, however the same network_cli would be used if you were talking to any other network device, such as a Dell PowerConnect switch, Cisco ASA firewall, PaloAlto device.

ansible_network_os: This variable tells ansible what SPECIFIC network device you are communicating with, in this example we are talking with a Cisco IOS 3850, so when specify the ios it’s telling ansible to use the IOS modules in the next task.

ansible_user: This is the username you would normally type in if you were to SSH to the switch.

ansible_ssh_pass : This is the password you would normally type if you were to SSH to the switch

ansible_become: This tells ansible if you need to escalate to the enable console, you are allowed to

ansible_become_method: This tells ansible what method it should use to elevate to the priv mode

ansible_become_pass: This is the password ansible should use when it attempts to elevate to priv mode

Next let’s throw together a playbook that will use the IOS_banner to set the login banner of the switch.

- hosts: all
    - /etc/ansible/group_vars/ios.yml
    - name: Setting the login banner on switch
       banner: login
       state: present

This is going to load our previously created ios.yml file that has all the connection variables that are needed when talking to a network device. Then it will use those variables to run the IOS_banner module.

Now, the last thing we need to do is create an IOS inventory file, if you have an existing inventory file you can just make a new [ios] group or you can make a new ios.ini file. I will simply make a new ios.ini file.

 ansible]$ touch ios.ini
ansible]$ vi ios.ini
(add my ip for this switch)

Now if we tie this all together in a single command, it will execute our playbook from the newly created inventory file, keep in mind this inventory file can be 1 or 100s of IOS devices, in this case it’s just 1 switch.

 ansible]$ ansible-playbook -i ios.ini playbooks/ios_test.yml

[WARNING]: Skipping plugin (/usr/lib/python2.7/site-
packages/ansible/plugins/connection/accelerate.py) as it seems to be invalid:
cannot import name key_for_hostname
/usr/lib/python2.7/site-packages/requests/__init__.py:80: RequestsDependencyWarn                                                                                                                                                             ing: urllib3 (1.22) or chardet (2.2.1) doesn't match a supported version!

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
[WARNING]: Ignoring timeout(10) for ios_facts
[WARNING]: default value for `gather_subset` will be changed to `min` from
`!config` v2.11 onwards
ok: [x.x.x.x]

TASK [Setting banner MOTD on switch] *******************************************
ok: [x.x.x.x]

PLAY RECAP *********************************************************************
x.x.x.x                   : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Please disregard the warnings, these are due to the fact that I am using a newer ansible version.

If your PLAY RECAP shows change=1 or more, then that means the play was a success and next time you login to your switch you’ll have a new banner!

You can continue to build playbooks now and load the var_file we created at the beginning of this to make more advance and indepth playbooks to begin down the path of automating your network infrastructure.

More details about IOS modules, network_cli options & variables can be found below




Get your Fabric Interconnects talking!

This post will walk thru configuring Fabric Interconnects. Read this after you have racked & cabled Fabric Interconnects. Before you begin, make sure you have the following available:

  • 3x IP Addresses (used for each FI and 1 VIP)
  • Cable L1,L2 and Mgmt ports on each FI
  • DNS Address(s)

We will walk thru initial configuration via the GUI web interface of the FI & console config of the secondary FI.

Connect to the console port of your primary FI and open your console session to begin. If you happen to connect to the console before you apply power, you’ll have the opportunity to watch the FI boot. If you watch closely you’ll notice an NXOS underpinning, it will look something like this.

Version 2.00.1201. Copyright (C) 2009 American Megatrends, Inc.
Booting kickstart image: bootflash:/installables/switch/ucs-6100-k9-kickstart.5.0.3.N2.4.02b.bin...................................................................................
.............................................................Image verification
 OKUsage: init 0123456SsQqAaBbCcUu
INIT: [   10.975503] I2C - Mezz present
Starting system POST.....
  Executing Mod 1 1 SEEPROM Test:...done (0 seconds)
  Executing Mod 1 1 GigE Port Test:....done (32 seconds)
  Executing Mod 1 1 PCIE Test:.................done (0 seconds)
  Mod 1 1 Post Completed Successfully
POST is completed
can't create lock file /var/lock/mtab~208: No such file or directory (use -n flag to override)
S10mount-ramfs.supnuovaca Mounting /isan 3000m
Mounted /isan
Creating /callhome..
Mounting /callhome..
Creating /callhome done.
Callhome spool file system init done.
nohup: redirecting stderr to stdout
autoneg unmodified, ignoring
autoneg unmodified, ignoring
Checking all filesystems..... done.
Checking NVRAM block device ... done
The startup-config won't be used until the next reboot.
Loading system software
Starting the smart check..
Uncompressing system image: bootflash:/installables/switch/ucs-6100-k9-system.5.0.3.N2.4.02b.bin

27+1 records in
27+1 records out
20480 bytes (20 kB) copied, 8.9105e-05 s, 230 MB/s
ethernet end-host mode on CA
FC end-host mode on CA
n_port virtualizer mode.
INIT: Entering runlevel: 3
touch: cannot touch `/var/lock/subsys/netfs': No such file or directory
Mounting other filesystems:  mount: /dev/hd-usbslot1 is not a valid block device
touch: cannot touch `/var/lock/s
/isan/bin/muxif_config: fex vlan id: -f,4042
Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Added VLAN with VID == 4042 to IF -:muxif:-

Changing of vsh_perm
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
enabled fc feature
System is coming up ... Please wait ...
2019 Sep 20 15:52:51  %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files begin  - clis
2019 Sep 20 15:52:53  %$ VDC-1 %$ Sep 20 15:52:53 %KERN-0-SYSTEM_MSG: [   10.975503] I2C - Mezz present  - kernel
System is coming up ... Please wait ...
System is coming up ... Please wait ...
2019 Sep 20 15:53:01  %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: loading cmd files end  - clis
2019 Sep 20 15:53:01  %$ VDC-1 %$ %USER-2-SYSTEM_MSG: CLIS: init begin  - clis
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
System is coming up ... Please wait ...
2019 Sep 20 15:54:31  %$ VDC-1 %$ %VDC_MGR-2-VDC_ONLINE: vdc 1 has come online
System is coming up ... Please wait ...
nohup: appending output to `nohup.out'
2019 Sep 20 15:54:53 switch %$ VDC-1 %$ %USER-2-SYSTEM_MSG: Running in PIO stats mode  - carmelusd

If you smash enter a few times on the console screen you’ll get this.

           ---- Basic System Configuration Dialog ----

  This setup utility will guide you through the basic configuration of
  the system. Only minimal configuration including IP connectivity to
  the Fabric interconnect and its clustering mode is performed through these steps.

  Type Ctrl-C at any time to abort configuration and reboot system.
  To back track or make modifications to already entered values,
  complete input till end of section and answer no when prompted
  to apply configuration.

   Switch can now be configured from GUI. Use and click
   on 'Express Setup' link. If you want to cancel the configuration from GUI and go back,
   press the 'ctrl+c' key and choose 'X'. Press any other key to see the installation progress from GUI

I plan to show both methods of the configuration, the GUI method for the primary FI and the console for the subordinate.

If you take note of the code from the snippet above, you’ll see an IP address. You will need to set your local IP to an address within that same subnet, then connect the computer to the MGMT port on the FI, once that is done you can browse to the FI GUI.

I set my local IP to

Now, connect to the FI GUI and you should get something like this

Open the URL and continue past any security alerts, the next page will take you to the beginning of the ‘Express Setup’

After you’ve clicked Express Setup, you will be presented with a very minimal configuration screen. This is where the IP Addresses mentioned earlier will come in handy.

I will be continuing with a clustered configuration. Select Fabric A as this is your first FI in the cluster. Then fill in the remaining information:

  • Virtual Address (address of the cluster)
  • System Name
  • Admin password
  • Mgmt IP Address (The individual IP of THIS FI)
  • Mgmt IP Netmask
  • Gateway
  • DNS Server
  • Domain Name (optional)
  • UCS Central IP & Shared Secret (optional)

Once all of the details have been filled in, click SUBMIT and your config will begin writing to the FI, you will be presented with the following:

If you switch back to the console you’ll see this on the console:

You can now login with the credentials of admin : (the password you just set)

AWESOME, HALF OF THE CONFIG IS DONE!! The second half we will complete via the console, and you will only need 1 piece of information.

  • Password of the primary FI
  • IP Address of the secondary FI (the one you are about to config)

Move your console cable from the primary FI, to the secondary and smash enter a few times. Your console should read something like this:

Type 'reboot' to abort configuration and reboot system or Type 'X' to cancel GUI configuration and go back to console  or Press any other key to see the installation progress from GUI (reboot/X) ?

Type X and hit Enter, next you will be asked to configure via GUI or Console, type Console and hit Enter.

Enter the configuration method. (console/gui) ? console

This is where you should have already connected your L1,L2 cables, if done correctly the FI will detect it’s peer and prompt you to enter the password of the primary FI.

Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the cluster. Continue (y/n) ? y

  Enter the admin password of the peer Fabric interconnect:
    Connecting to peer Fabric interconnect... done
    Retrieving config from peer Fabric interconnect... done
    Peer Fabric interconnect Mgmt0 IPv4 Address: X.X.X.X
    Peer Fabric interconnect Mgmt0 IPv4 Netmask:
    Cluster IPv4 address          : X.X.X.X

    Peer FI is IPv4 Cluster enabled. Please Provide Local Fabric Interconnect Mgmt0 IPv4 Address

  Physical Switch Mgmt0 IP address : X.X.X.X

Hit Enter and you will be asked to save the config, type Yes to save the config

Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes
  Applying configuration. Please wait.

Fri Sep 20 16:09:40 UTC 2019
  Configuration file - Ok

Cisco UCS 6200 Series Fabric Interconnect
HX-Production-B login:

Thats all! You have now configured BOTH FIs! If this made no sense to you, please find more information on Cisco’s UCS Platform and their Fabric Interconnects using the link below. In simple terms, the Fabric Interconnects allow you to virtualize your hardware. If that sounds cool you should also click the link to learn more.


Who am I?

As technology evangelist with 10+ years of hands-on experience, I bring an innovative and pragmatic approach to analyzing complex business needs. I am able to conceptualize, design, and implement cutting edge solutions based on the latest virtualization and cloud technologies. 

Why am I doing this?

Information should be free and available to everybody. That’s the idea behind this blog.

Manuals are great, but can be a lot of digging and hunting for a specific answer or picture of a problem you’re working on. This will bridge that gap.

In my day job, I get to play with lots of cool technologies and I want to take some extra time to document it and share the wealth of knowledge with the community.

A few key topics I will be focusing on:

  • Automation
  • Deployments of new and emerging enterprise technologies
  • Things that break and why
  • Old and outdated technologies
  • Anything cool