Archive for category Cisco

EEM Event Detectors

In my previous post, I stated that EEM has these Event Detectors which are used to catch the events and then trigger policies which you have created.  So the ability to do anything with EEM relies greatly upon the detectors themselves.  After all, if you want to be able to do something (with a policy), you have to be able to detect the trigger.  It would be a little like having a policy to open the door for someone when they arrived at your doorstep.  The policy would be opening the door but without the ability to detect whether or not someone was at your door, you’d have a hard time.  The detector in this case, determining whether or not someone was at your door, is critical to the entire operation.  Such is the case with the EEM event detectors.  So what are they?  These are the list of event detectors supported by the various versions of Cisco IOS.

Event Detector EEM v2.2/2.3 EEM v2.4 EEM v3.0
Syslog x
SNMP x
Watchdog x
Counter x
Interface x
Timer x
Application Specific x
OIR x
CLI x
GOLD x
Resource x
Redundancy Framework x
Enhanced Object Tracking x
None x
EEM-RPC x
SNMP Proxy x
NetFlow x
IP SLAs x
Routing x
Custom CLI x

So what are these and what can they do?  Here’s a brief comment about each one but I’ll talk about each one in depth in future posts. Some of them are more common than others.  For example, I done think that many people use the GOLD ED but Syslog and CLI are extremely popular.

Syslog – Monitors the syslog messages sent by the device.  You can use this to match using a regular expression and act accordingly.

CLI – This ED is used everytime a CLI command is issued by a user.  What’s great about this ED is that you can configure it so that you can suspend the CLI command that the user issued until the completion of the policy.  This means that you can use your policy to disallow access to that CLI command.

Timer – This ED is used to setup a countdown (and stop) timer, a countdown (and restart counting down) timer, a specific time,  or a recurring interval timer.  The coundown (and stop) timer acts like a kitchen egg timer.  After a specific number of seconds, the EEM server fires off the associated policy.  The timer is NOT restarted with this timer. There is atimer that does support that type of functionality.  In essence, it counts down, fires off the policy and starts counting down again.  So the end result is, you can have something fire every 42 seconds, for example.   Third, there is a specific time timer which fires off at a specific time, say, August 23, 2009 at 9:03am.  The fun with this one is that you have to configure the time in number of seconds past midnight January 1, 1970.  (Use an online conversion utility like this one).  Finally, there’s a timer which behaves just like the UNIX cron utility (or Cisco’s Kron).

None – This ED is not triggered by any event that happens within IOS.  It is used to be triggered by hand or by another script.

SNMP – This ED polls a specific OID.  The value can then be processed and a policy can act accordingly.

InterfaceThis ED is used to process the interface counters which can be seen when you type ’show interface’.  There are 22 some odd counters that can be monitored with this ED.

Counter – Along with the None ED, this is another detector that can be used to trigger another policy.  The counter is a value that can be manipulated from within the policies and referenced by other policies.  For instance, you can have one policy update the counter and have another policy (which uses the counter ED) rely upon the counter value set previously to trigger.

Watchdog/WD System Monitor - These two EDs monitor the CPU and memory within the device.  The Watchdog ED is used for non-modular code and the WD System Monitor is used on the new modular code.  This is handy when you are concerned about processes using too much CPU or memory.

OIR (Online Insertion and Removal) – This ED triggers upon the insertion and removal of cards.  For example, if a line card in a 6500 chassis was removed and replaced, this ED would trigger.

Application Specific – This ED is used by a policy to publish an event which can then be used to trigger another event.

Generic Online Diagnostic (GOLD) - The Generic Online Diagnostic (GOLD) is a framework for diagnotics which is currently implemented in some Cisco IOS products.  It is designed to help detect hardware problems that can occur on devices.  The results of these tests can be captured using the GOLD ED.

Resource – Cisco has a feature which allows a more in-depth view of the allocation of resources such as CPU, memory and buffers called the Embedded Resource Manager.  This utility is able to generate events such that they can be captured by the Resource ED.

Redundancy Framework -The Redundancy Framework is a feature which provides high availablity to various Cisco devices, such as the Cisco 6500 series switches.  It is one of the elements that allows the device to support NSF/SSO functionality.  The redundancy framework generates events which can be detected by the redundancy framework ED.

Enhanced Object Tracking – Yes, the vary same feature that allows you to track IP SLA statistics, routes, track lists, etc.  These events can be detected by the Enhanced Object Tracking ED.

EEM-RPC – This event detector can accept SOAP commands over SSHv2.  The commands are formatted as XML code which tell the event detector to fire off an EEM policy.

SNMP Notification – This event detector can detect SNMP trap messages which are sent to the device.  This allows the ability to peform an action (through a policy) in response to an SNMP trap detected from another system

NetFlow – Supports Flexible NetFlow which allows the detection of various flow attributes such as the destination IP address or port number.  It also allows the detection of flow rate so the ED can trigger if the flow rate exceeds a specific threshold.

IP SLAs – This event detector supports the IP SLA functionality that is available within IOS.  It allows the detection of changes in the failing or meeting of a programmed SLA.

Routing – Allows the detection of Routing Information Base (RIB) events such as the addition, remove or modification of a route.

Custom CLI – Allows the creation of custom CLI commands which allow the addtion of special characters such as ‘?’ or ‘Enter’.

No Comments

Embedded Event Manager Beginnings

Cisco’s Embedded Event Manager is probably the most under-utilized ability of the Cisco IOS.  It has been available for sometime time and is built into almost all modern IOS based products.  Notice that I stated IOS based, this excludes ASA, PIX, and some of the more esoteric Cisco products.  But it is available on switches and routers (including the XR line).  Here’s how it works:

In the normal operations of the switch or router, events happen all the time.  These events can be someone issuing a CLI command, a syslog message, an SNMP trap, etc.    As these things happen, they can be detected by the EEM Event Detectors, which are running all the time in the background.  The Event Detectors send their information to the EEM server. Here’s a picture from the Cisco website

datasheet_c78-492444-1

The EEM server can then be programmed to fire off an EEM policy.  The policy is what you can program.  Currently there are two different types of polices:  applets and Tool Command Language (Tcl) scripts.  These applets and Tcl scripts can be programmed to do all sorts of fun things.  They can send syslog messages, SNMP traps, fire off e-mails, even open raw sockets.  That’s right, you read it right, you can open raw sockets! If you’re not familiar with Tcl, get familar.  It’s a lot like Perl and Python just a new syntax but it has them same power.   Let that soak it, you have a script interpreter built into your router!

Think about the power you have now.  You can program your router to detect the user typing a specific CLI command, stop them, and fire off an e-mail in response.  You can capture the MAC address table every five minutes, detect any changes, and syslog it off. To get a better feeling about what people are doing out there, check out the Cisco Beyond page.

I plan on writing a lot about this in the future so expect Tcl scripts, applets, discussions about how to all sorts of things.  Let me know if there’s something you’d like to see.

1 Comment

Multicast Question #4 – Answered

The fourth question was:

* What happens when the receiver decides it wants to bypass the RP for group 239.0.1.23 and receive the multicast stream directly from the host?

Answer: So this is a pretty easy one (which makes me feel dumb for not writing it sooner).  By default, in sparse mode, a Cisco router with an attached listener will attempt to form a source-path tree (from the root-path tree) after receiving the first packet of data from the source.   Essentially, this means that in sparse mode, the receivers rely upon the RP to be the root of a tree which sends the stream of data that it’s interested in.  This is because it doesn’t really know where the sender is coming from (it could be unknown or from multiple places).  Once the router attached to the receiver realizes where the source of the stream is coming from, it can get it directly from the source.

How does this work?  Glad you asked.  Take a look at the following diagram.

blog_ans4

Router B is going to be sending the data from the Source to the RP.   But look at the RP! The root path three is highlighted in red. It doesn’t make much sense for the Host to receive this data in this roundabout manner.  Why can’t Router A just get the stream from Router B?  That’s exactly what happens when Router A realizes that it received the data from an interface that isn’t consistent with it’s routing table.  Imagine that you could view Router A’s routing table.  It would show that the shortest path to get to the Source would be through Router B.  So Router A sends a PIM Join message to Router B which allows it to bypass the RP.  Now the flow follows the highlighted blue line. Much more efficient.

Can this behaviour be changed?  Absolutely.  The command (global) to use is:

ip pim spt-threshold <rate (in Kbps)>

When the configured rate is exceeded, the change to a SPT will be made.  If you send it to 0, the change will never occur.

No Comments

Multicast Question #3 – Answered

The third question was:

* If the host is sending data for group 239.0.1.23 (and sparse-mode is being used), which router (A or B) is going to forward the stream to the RP?

Answer: If you take a look back at the answer to Multicast Question #1, the host will send an IGMP membership report using the destination address of the group (in this case, 239.0.1.23).  If both router A and B are listening to these IGMP messages, they will both receive the same membership report.  It is then the job of the router (A or B) to send the PIM Register message to the RP.  Assuming that there are hosts out there somewhere who want to receive the stream, the RP will accept the PIM Register message and the stream will head toward the RP.  Of course, the question was, WHICH router?!  The answer is the same as it was in Question #1.  The PIM DR for the segment will be the forwarder.

After firing up GNS3, we run a test:

R1#show ip mroute
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
       L - Local, P - Pruned, R - RP-bit set, F - Register flag,
       T - SPT-bit set, J - Join SPT, M - MSDP created entry,
       X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
       U - URD, I - Received Source Specific Host Report,
       Z - Multicast Tunnel, z - MDT-data group sender,
       Y - Joined MDT-data group, y - Sending to MDT-data group
Outgoing interface flags: H - Hardware switched, A - Assert winner
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.0.1.23), 00:03:23/stopped, RP 192.168.0.1, flags: SJPCF
  Incoming interface: FastEthernet0/1, RPF nbr 192.168.0.1
  Outgoing interface list: Null

(10.0.1.110, 239.0.1.23), 00:03:23/00:02:57, flags: FT
  Incoming interface: FastEthernet0/0, RPF nbr 0.0.0.0
  Outgoing interface list:
  FastEthernet0/1, Forward/Sparse, 00:00:07/00:02:52

Notice the flags that are set on the (*,G) entry.  The ‘C’ flag is set which indicates that there is a device connected to the router which belongs to the group.  In the (S,G) entry the RPF nbr of 0.0.0.0 means that it is connected to the sender.  It looks like our PIM DR theory proved true!

No Comments

Multicast Question #2 – Answered

The second question was:

* Which router (A or B) is going to send the queries to the LAN to determine if there are any more active listeners?

Answer:  Obviously, only one of the routers should query the LAN to determine if there are any listening hosts.  IGMP determines which router should send the queries on to the segment using the IP addresses.  If R2 sends it’s query first, R1 will hear the query and determine that R2 should be the active querier on the segment because R2’s IP address is lower.  The opposite could happen, and R1 might be active first, send the query and then R2 could come alive.  In this case, R2 would see that it has a lower IP address and send it’s query.  R1 would then hear the query from a lower IP address and reliquesh it’s duties to R2.   So what happens if R2 fails?  R1 has a timer called the ‘query timeout’.  If that time runs out and R1 hasn’t heard a query from R2, it takes control and starts to query.

No Comments

Cisco Live 2009 and CVOICE

I signed up to go to Cisco Live 2009, which I’m really excited about.  I plan on taking a lot of classes on EEM and TCL scripting for routers (look for posts about this in the future).  I’m also taking the CVOICE exam since it’s covered as part of the conference.  Needless to say, I’m going to be learning as much as I can, as fast as I can.  I really don’t think that I’ll pass because IP Telephony is so much different than traditional networking.  That’s part of why I want to get involved.  It’s a whole new world of networking!

No Comments

Multicast Question #1 – Answered

So the first question was:

* If the host is receiving data for group 239.0.1.23, which router (A or B) is forwarding the data to SW1 and the host?

Answer: Let’s consider the Sparse Mode case first. In Sparse Mode, the receivers must express their interest in joining the group through IGMP. So the answer to this is simply the router to which the IGMP packet was sent to. It is this router that sends the PIM join message toward the RP. So which router does the host send the IGMP packet to? When a host decides that it wants to join a multicast stream, it sends an IGMP membership report. The destination address that it uses to send the membership report to is the group address that the host wishes to join. So we still haven’t answered the question! Time for alittle experimentation with GNS3.

R1#show ip mroute
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
L - Local, P - Pruned, R - RP-bit set, F - Register flag,
T - SPT-bit set, J - Join SPT, M - MSDP created entry,
X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
U - URD, I - Received Source Specific Host Report,
Z - Multicast Tunnel, z - MDT-data group sender,
Y - Joined MDT-data group, y - Sending to MDT-data group
Outgoing interface flags: H - Hardware switched, A - Assert winner
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode
(*, 239.0.1.23), 06:52:35/00:02:05, RP 0.0.0.0, flags: SJC
Incoming interface: Null, RPF nbr 0.0.0.0
Outgoing interface list:
FastEthernet0/0, Forward/Sparse, 00:01:05/00:02:05

R1 is forwarding group traffic out toward the host through interface Fa0/0.

R2#show ip pim neighbor
PIM Neighbor Table
Mode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority,
S - State Refresh Capable
Neighbor                      Interface                                   Uptime/Expires              Ver   DR
Address                                                                                                                                                                             Prio/Mode
10.0.1.254                 FastEthernet0/0    07:05:52/00:01:31     v2    1 / DR S

Why did R1 get chosen? Look at the PIM DR for the segment. Remember that the PIM DR is based upon two values:

  1. The DR Priority (the default is 1 and highest wins)
  2. The IP address (highest IP address wins)

So the DR by default would be R1 because it’s IP address is higher. To test that theory, let’s set the priority of R2 to 100 so it becomes the DR.

R2#config t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#int f0/0
R2(config-if)#ip pim dr-priority 100

Now look at the PIM Neighbors on both R1 and R2 to verify that R2 is now the DR:

R1#show ip pim neighbor
PIM Neighbor Table
Mode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority,
S - State Refresh Capable
Neighbor          Interface                                Uptime/Expires           Ver            DR
Address                                                     Prio/Mode
10.0.1.253             FastEthernet0/0              00:10:25/00:01:41     v2               100/ DR S
R2#show ip pim neighbor
PIM Neighbor Table
Mode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority,
S - State Refresh Capable
Neighbor           Interface                    Uptime/Expires                Ver           DR
Address                                                                                                                                                                          Prio/Mode
10.0.1.254               FastEthernet0/0        00:10:20/00:01:43     v2      1 / S

R2 is now the DR and has a priority of 100! The final step is to verify that it’s forwarding traffic for the group:

R2#show ip mroute
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
L - Local, P - Pruned, R - RP-bit set, F - Register flag,
T - SPT-bit set, J - Join SPT, M - MSDP created entry,
X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
U - URD, I - Received Source Specific Host Report,
Z - Multicast Tunnel, z - MDT-data group sender,
Y - Joined MDT-data group, y - Sending to MDT-data group
Outgoing interface flags: H - Hardware switched, A - Assert winner
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode
(*, 239.0.1.23), 00:04:16/00:02:17, RP 0.0.0.0, flags: SJC
Incoming interface: Null, RPF nbr 0.0.0.0
Outgoing interface list:
FastEthernet0/0, Forward/Sparse, 00:04:16/00:02:17

R2 is indeed now forwarding group traffic from 239.0.1.23 out F0/0 toward the host.

What about the Dense Mode solution, you cry? Remember that in Dense Mode, the routers just spew the data out all of their multicast interfaces. So assuming that R1 and R2 both get added to the tree, they will both want to send data out their Fa0/0 interfaces toward the host. This of course makes the assumption that the Fa0/0 interface on both R1 and R2 pass the RPF check (more on that to come).  PIM Dense Mode is smart enough to realize that both routers sending the multicast stream into the LAN segment means that each host will receive multiple copies of the same data.  To stop this, the PIM routers detect each other and decide on one router to forward the stream on the segment using Assert messages.  Essentially, the routers decide which router has the lowest administrative distance to the source.  In the event that they have same administrative distance (which is likely), the lowest metric in that unicast routing protocol wins.  If they have the same metric, the router with the highest IP address wins.  Thus, R1 would win.   The winner of the Assert process can be seen with an ‘A’ in the ’show ip mroute’ command.

(192.168.0.1, 239.0.1.23), 00:00:02/00:02:57, flags: T
Incoming interface: FastEthernet0/1, RPF nbr 0.0.0.0
Outgoing interface list:
FastEthernet0/0, Forward/Dense, 00:00:02/00:00:00, A

No Comments

Multicast Question

Given the following diagram, answer the following questions:

* If the host is receiving data for group 239.0.1.23, which router (A or B) is forwarding the data to SW1 and the host?
* Which router (A or B) is going to send the queries to the LAN to determine if there are any more active listeners?

* If the host is sending data for group 239.0.1.23 (and sparse-mode is being used), which router (A or B) is going to forward the stream to the RP?
* What happens when the receiver decides it wants to bypass the RP for group 239.0.1.23 and receive the multicast stream directly from the host?

Answers coming soon….

No Comments

Weighted CCIE Blueprint

So I finally finished the weighted CCIE Blueprint. Just to re-iterate, this is based on my experience with taking numerous practice tests.  If you disagree with some or all of these.  Let me know, let’s discuss and make some changes.  This is ultimately a living document.  Maybe there’s some stuff that I don’t have on here that should be.

Download the weigthed CCIE Blueprint

No Comments

Weighted CCIE Lab Blueprint

What I’ve decided to do is to use the Internetwork Experts CCIE Lab Blueprint and modify it to show what I believe is important (at least relative to other tasks).  This is based upon my cumulative experience with practice tests, etc.  This is NOT based upon my test!  I hope to have it posted here in a few days.  Stay tuned.

No Comments