Monday, August 6, 2012

CCIE-V Tip: Using Config Replace

Seeing that I have not taken the practical yet, it may be a bit presumptuous of me to offer up "tips" on taking the CCIE-V lab. Maybe it is better to think of this post as more of an idea I have or a process I plan to adopt when taking my own lab. I invite others to comment if they have suggestions for improvement.

I feel that when you are entering a pressure cooker, it is best to have a well thought out plan of execution. For the CCIE-V lab you can't have a detailed plan for every situation. There are just too many permutations. However, you can (and should) have some "base" methodologies that you attach to your test taking strategy. I think one of these base plans should include configuration management.

The problem I am focused on here is making sure I have a process to roll back configurations during the middle of the lab -- just in case I break something.  I was inspired to think on this problem after watching some of the excellent videos from Kevin Wallace. I suspect that my idea is nothing at all new for veterans who are preparing for this lab. The tools I will be discussing are staples for anyone who has been studying on their own lab gear or managing live systems.

Background

As I started working through my own self-study for the CCIE-V I needed a method for quickly resetting my lab systems to a relatively blank configuration, while also giving me a way to jump back (or forward?) to a lab scenario to review topics where I need some polishing. 

This line of thinking brought me to the archive feature and, more importantly, the rollback feature offered by config replace. I incorporated these tools into my lab "maintenance strategy" (for lack of a better term) and they are serving me well. It is now a relatively simple task to jump from one configuration to another.

Until recently, I was only thinking of the archive/config replace duo as tools I use to keep my lab configurations organized. A week ago, I was watching a video posted by Kevin Wallace where he provided some tips on taking the IE voice exam. One of his tips was to save configurations on your lab switch(es) and routers to notepad. The idea being to have a method to quickly look back at an old configuration just in case you feel like you have lost your way.

I think Kevin's idea is sound and the need he has identified is real. However,  I am presently thinking that it may be slightly better to archive configurations locally on the routers flash and then incorporate built-in archive and config replace features into the test taking methodology. I say this knowing that it is very possible I may change my mind. Though, I have actually used the methods I discuss herein to back out of a bad config (or two).

Configuration Management Tools

Before defining a specific strategy it is a good idea to understand the basic tools that are available. Once you get a handle on these tools then you can think of ways to use them that are more in tune with your test taking style. This is my fundamental philosophy on all things in our industry: Listen to what others say about "best practice". Understand the fundamental moving parts of this "best practice" guidance/opinion. Develop your own approach based on this understanding and your individual experience. (Re-)Evaluate your approach on a regular basis to see if it is still relevant.

Let's get to it, shall we?

The Archive Feature/Facility

While I don't think I will use the actual archive feature in my lab process, I came upon the various tools I do plan to use by way of the archive feature. I also think that others may find it quicker/easier to simply use the archive feature directly.

IOS includes a configuration archive feature that allows for both manual and automated configuration snapshots, which can be stored locally on the IOS device's flash system. Enabling this feature is pretty straightforward. You create an archive directory (recommended) and simply tell the IOS device where this archive directory is located.

HQ-RTR# mkdir archive
Create directory filename [archive]? 
Created dir flash:archive
HQ-RTR# dir
Directory of flash:/

    1  drw-           0   Jul 3 2012 13:11:20 +00:00  ios-bin
    2  drw-           0   Jul 3 2012 21:01:34 +00:00  archive
    3  -rw-      496521  Jul 21 2012 19:27:16 +00:00  music-on-hold.au
    4  -rw-    53661736   Jul 3 2012 13:02:12 +00:00  c2800nm-adventerprisek9_ivs_li-mz.124-15.T9.bin

! now you can turn on archive
HQ-RTR#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
HQ-RTR(config)#archive
HQ-RTR(config-archive)#path flash:/archive/

If you decide to not use the archive feature as part of your lab methodology, you do not need to configure it. You can still use most of the commands usually associated with the archive feature (e.g config replace, show archive config) without enabling archive.

Archiving the Configuration

There are two methods that you can use to "archive" configurations on a device. You can use the archive facility or your can save your configuration and then copy it to a backup file. Let's look at the archive facility first using the following example.

The first step is to know the limits. The archive facility will save up to 10 configurations by default. You can increase (or decrease) this parameter. The maximum value is 14.

HQ-RTR#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
HQ-RTR(config)#archive
HQ-RTR(config-archive)#maximum 14

Now, let's take a look at the archive before and after archiving a configuration. We can view the archive using the show archive command. Each slot will show the name (including path) of the archive file. On initial configuration, there are no configurations in the archive.
 
HQ-RTR# show archive
There are currently no configuration saved.
The next archive file will be named flash:/archive/-0
 Archive #  Name
   1         
   2         
   3         
   4         
   5         
   6         
   7         
   8         
   9         
   10        
   11        
   12        
   13        
   14        
HQ-RTR#

We can archive the configuration using the archive config command. After archiving a configuration, you can then use the show archive command to see the new archive entry. When you have multiple entries, IOS will indicate the most recent entry. You can also see the actual archived configuration file by using the dir command and looking in the directory you designated as the archive path (flash:/archive/ in our example).
 
HQ-RTR#archive config

HQ-RTR#show archive
There are currently 1 archive configurations saved.
The next archive file will be named flash:/archive/-1
 Archive #  Name
   1        flash:/archive/-0 <- Most Recent
   2         
   3         
   4         
   5         
   6         
   7         
   8         
   9         
   10        
   11        
   12        
   13        
   14        
HQ-RTR# dir flash:/archive/
Directory of flash:/archive/

   19  -rw-        5083   Aug 6 2012 14:12:08 +00:00  -0
    3  -rw-        1988   Jul 9 2012 04:09:28 +00:00  base-startup-config
    4  -rw-        2406  Jul 15 2012 17:18:04 +00:00  owle-lab1-startup-config
    5  -rw-        8856  Jul 22 2012 04:55:54 +00:00  WrkBook2Lab1-startup-config-end
    6  -rw-        2244  Jul 28 2012 19:15:46 +00:00  wrkbk1-lab3-final-startup-config
    7  -rw-        4379  Jul 29 2012 01:55:46 +00:00  wrkbk1-lab4-final-startup-config
    8  -rw-        5508  Jul 31 2012 06:29:24 +00:00  wrkbk1-lab5a-final-startup-config
    9  -rw-        5181   Aug 4 2012 04:49:14 +00:00  wrkbk1-lab5c-final-startup-config

128147456 bytes total (69234688 bytes free)

Another method that can be used to archive the configuration is to simply copy the startup-config from NVRAM to FLASH. This is the method I started using in my home lab once I had enough gear to move away from using the Proctor Labs full time.

Example:
HQ-RTR#wr mem
Building configuration...

[OK]
HQ-RTR#copy nvram:startup-config flash:/archive/sample-startup-config
Destination filename [/archive/sample-startup-config]?

5095 bytes copied in 0.560 secs (9098 bytes/sec)
HQ-RTR#dir archive
Directory of flash:/archive/

   19  -rw-        5083   Aug 6 2012 14:12:08 +00:00  -0
    3  -rw-        1988   Jul 9 2012 04:09:28 +00:00  base-startup-config
    4  -rw-        2406  Jul 15 2012 17:18:04 +00:00  owle-lab1-startup-config
    5  -rw-        8856  Jul 22 2012 04:55:54 +00:00  WrkBook2Lab1-startup-config-end
    6  -rw-        2244  Jul 28 2012 19:15:46 +00:00  wrkbk1-lab3-final-startup-config
    7  -rw-        4379  Jul 29 2012 01:55:46 +00:00  wrkbk1-lab4-final-startup-config
    8  -rw-        5508  Jul 31 2012 06:29:24 +00:00  wrkbk1-lab5a-final-startup-config
    9  -rw-        5181   Aug 4 2012 04:49:14 +00:00  wrkbk1-lab5c-final-startup-config
   20  -rw-        5095   Aug 6 2012 14:37:36 +00:00  sample-startup-config

128147456 bytes total (69226496 bytes free)

Again, this isn't using the "archive" feature. It is simply saving a configuration and then manually making a copy of that configuration.

Comparing Configurations

One of the features you can use that is related to the archive facility is the show archive config differences command. You can use this command on configuration archives that are created using the archive config command AND on configuration files that you manually backed up. Let's look at an example.

HQ-RTR(config)#do sh runn int gig 0/0.20
Building configuration...

Current configuration : 203 bytes
!
interface GigabitEthernet0/0.20
 description IELab-HQ-PHONES
 encapsulation dot1Q 20
 ip address 10.3.200.3 255.255.255.0
 ip helper-address 10.3.120.10
 h323-gateway voip bind srcaddr 10.3.200.3
end

HQ-RTR(config)#int gig 0/0.20
HQ-RTR(config-subif)#description CHANGE.ME
HQ-RTR(config-subif)#^Z
HQ-RTR#wr mem
Building configuration...

[OK]
HQ-RTR#archive config


HQ-RTR#show archive
There are currently 3 archive configurations saved.
The next archive file will be named flash:/archive/-3
 Archive #  Name
   0       flash:/archive/-0
   1       flash:/archive/-1
   2       flash:/archive/-2 <- Most Recent
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14

HQ-RTR#show archive config diff flash:/archive/-1 flash:/archive/-2
Contextual Config Diffs:
interface GigabitEthernet0/0.20
 +description CHANGE.ME
interface GigabitEthernet0/0.20
 -description IELab-HQ-PHONES


In the example above, we are comparing two files that were created using the archive facility. However, the show archive config diff command is pretty flexible and will allow you to compare any two configuration files, whether they are stored on the local system or remotely (i.e. ftp, http, tftp, etc.). For example, let's say we are doing the QoS section and we need to provision the HQ router frame-relay connection to BR1-RTR for QoS. 

HQ-RTR#copy nvram:startup-config flash:/archive/pre-auto-qos-config
Destination filename [/archive/pre-auto-qos-config]?

5083 bytes copied in 0.480 secs (10590 bytes/sec)
HQ-RTR#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
HQ-RTR(config)#int se 0/3/0.1
HQ-RTR(config-subif)#bandwidth 384
HQ-RTR(config-subif)#frame-relay interface-dlci 122
HQ-RTR(config-fr-dlci)#auto qos voip trust
%Creating new map-class.
HQ-RTR(config-fr-dlci)#^Z
HQ-RTR#wr mem
Building configuration...

[OK]
HQ-RTR#copy nvram:startup-config flash:/archive/post-auto-qos-config
Destination filename [/archive/post-auto-qos-config]?

5970 bytes copied in 0.420 secs (14214 bytes/sec)
HQ-RTR#
HQ-RTR#show archive config diff flash:/archive/pre-auto-qos-config flash:/archive/post-auto-qos-config
Contextual Config Diffs:
+class-map match-any AutoQoS-VoIP-RTP-Trust
 +match ip dscp ef
+class-map match-any AutoQoS-VoIP-Control-Trust
 +match ip dscp cs3
 +match ip dscp af31
+policy-map AutoQoS-Policy-Trust
 +class AutoQoS-VoIP-RTP-Trust
  +priority percent 70
 +class AutoQoS-VoIP-Control-Trust
  +bandwidth percent 5
 +class class-default
  +fair-queue
interface Serial0/3/0
 +frame-relay traffic-shaping
interface Serial0/3/0.1 point-to-point
 +bandwidth 384
 frame-relay interface-dlci 122
  +class AutoQoS-FR-Se0/3/0-122
  +auto qos voip trust
 +frame-relay ip rtp header-compression
+map-class frame-relay AutoQoS-FR-Se0/3/0-122
--more--

As shown in the example above, you can compare files you have manually created. Also, since the nvram:startup-config is a file stored on the system (albeit in nvram) you can also use startup-config in the diff operation, for example:show archive config diff flash:/archive/some-archive-config nvram:startup-config

Viewing Configurations

For completeness, I wanted to provide a quick tip on how you can view archived configuration files. You do not need to use the show archive config diff command to look at an archived configuration file. You can easily use the more command to look at the raw configuration in any give file (e.g. HQ-RTR#more flash:/archive/some-archive-config).

You don't have to stop there either. You can pipe the more command to the other text parsing functions of IOS. For example: HQ-RTR#more flash:/archive/post-auto-qos-config | s Auto. Same idea applies for using "| include", "| begin", etc.


Rolling Back Configurations

Let's say you are in the lab. You have been cranking it out for 5+ hours and then you realize you have made a huge mistake. Maybe you accidentally saved your configuration while in SRST mode or maybe you royally botched up the QoS configs on the HQ-RTR. You are under pressure and while you may know the config sequence needed to undo what you just did, it may be easier/quicker just roll everything back.

Continuing with our QoS example. Let's say I want to roll back to my previous configuration. 
HQ-RTR#config replace flash:/archive/pre-auto-qos-config
This will apply all necessary additions and deletions
to replace the current running configuration with the
contents of the specified configuration file, which is
assumed to be a complete configuration, not a partial
configuration. Enter Y if you are sure you want to proceed. ? [no]: y


% Class-map AutoQoS-VoIP-RTP-Trust is being used
% Class-map AutoQoS-VoIP-Control-Trust is being used

*Aug  6 15:24:04.526: Rollback:Acquired Configuration lock.
*Aug  6 15:24:06.855: %PARSER-6-EXPOSEDLOCKRELEASED: Exclusive configuration lock released from terminal '0' -Process= "Exec", ipl= 0, pid= 3



Total number of passes: 2
Rollback Done

The following commands are failed to apply to the IOS image.
********
no frame-relay ip rtp header-compression
********


HQ-RTR#
*Aug  6 15:24:11.171: %PARSER-3-CONFIGNOTLOCKED: Unlock requested by process '3'. Configuration not locked.
HQ-RTR#

Sometimes you will see an error/warning in the rollback output such as the "...following commands are failed to apply..." message in the above output. This can get a little tricky. In my lab I have found that rolling back configurations on a running config is not always perfect. The first thing to do in this case is to check and see if there was a config element that wasn't cleaned up. So, after doing the rollback, save the configuration. Then compare the rollback file to the startup config.

HQ-RTR#wr mem
Building configuration...

[OK]
HQ-RTR#show archive config diff flash:/archive/pre-auto-qos-config nvram:startup-config
Contextual Config Diffs:
!No changes were found

HQ-RTR#

Well, there you go. No need to fret. The pre-auto-qos-config file and the startup-config file are the same and that means you have rolled back A-OK. In my lab, when I am resetting a configuration file or I have an issue with config rollback completed correctly, I will do the following:
  • wr erase
  • reload
  • Say "no" to config wizard dialog
  • config replace 
The above process is clean and works well when I am resetting my lab configuration to either the base config or to a specific lab state. When taking the actual lab I think that I will avoid that process if at all possible. It is time consuming. I am just sharing here so we have a complete picture.

Putting This Into Practice

Up to this point we have discussed the mechanics. The individual features that we can leverage. Putting this into practice is the big challenge. For lab preparation (i.e. maintenance of your home lab configs) there is no doubt in my mind that archiving configurations and using the config replace feature is 100% the way to go. I have found it very easy to jump around labs or jump to a specific point in any given lab so I can focus on furthering my understanding on a particular topic.

But how useful is this knowledge when sitting for the real lab? When I pose that question to myself, I go back to Kevin's discussion on making sure you save the current configurations to notepad before you start. I thought of that process and while I believe Kevin is dead-on in identifying the need, it just seems more productive to basically create an archive directory and do a copy nvram:startup-config flash:/archive/base-config. Then I have the base configuration to view at any time and I don't need to have a mostly unnecessary notepad instance floating around. I am not saying one method is better than the other at this point, just that I think there is a method that will work better for me.

Of course, I am still in the process of honing my approach. Trying to find the most efficient way to do things. Who knows, I may change my mind (and I reserve the right to do so!). Right now, I don't think I would use the actual archive config command. Which means I don't need to set up the archive feature. The only reason I have this predilection is because I think I would lose track of what config instance flash:/archive/-2 is actually referring to. For me, it is better to use flash:/archive/pre-qos-config (or similar name). Though, if you tracked your configs using your notepad or scrap paper, that could work.

Going back to the real point. The need we are trying to address here is to have a method to back out of a tough spot when we have done something stupid. In an 8 hour lab, it may be more efficient to hit a "reset" button then it is to try and undo/redo configurations one command at a time. The secondary point is to pick a method that works for you AND be willing to adjust if/when you find that your method is no longer optimal.  



Thanks for reading. If you have time, post a comment!

1 comment:

  1. Nice idea and I think this could be very useful as you could do this at various stages in the real lab, especially at the initial and before any major paste or configuration.

    ReplyDelete