Saturday, July 24, 2010

Book Review - Inside Cisco IOS Software Architecture

Ever wonder what the difference is between the CEF table and the Adjacency Table?
What is the default behavior for IOS if CEF is not enabled?
What is the difference between CEF and dCEF?

I got to read this book this week, and it is great for anyone who want to know more about the in-depth logic and guts of the IOS software. Some really great explanations here that outline the differences between Process Switching, Fast Switching, Optimum Switching, and CEF. It has some great graphics (flow charts mostly) to explain Load Sharing within CEF, and many other concepts. It explains how to troubleshoot CPU problems, and how they occur within the platform.

Personally, I found the most interesting parts where the author explains logical steps of different features native to the IOS platform. These are basically spread all around the platform, but particularly relevant are CEF and Fast Switching and how they are done in detail. The book also covers hardware architectures, I/O Memory, and gives insight into interpreting the output of "show diag", "show proc", "show interface stat", "show controllers", and other hardware-related diagnostic tools.

It also has a really good chapter on QoS where the author outlines queueing logic, different QoS technologies and how they are relevant to hardware, and finishes with NetFlow.

While the hardware platforms focused on in this book were aggregation-layer devices like the 7200 series routers, it would have been nice to read about the 3560 (which has powerful queueing architectures) and the 6500 series switches for example. Also, the material is a bit old, but valid nonetheless (don't expect to find anything on NX-OS or MPLS/Label Switching!)

You can buy the book from Amazon here

Monday, July 19, 2010

NSF/SSO for High-Availability deployments

Ever wonder what this output meant?

R1# show ip protocols
*** IP Routing is NSF aware ***

So I'm Currently reading up on NSF/SSO for HA campus design and MPLS PE networks.

I was actually reading up on OSPF LLS (RFC5613), when I discovered that Cisco uses the LLS field to implement NSF (Cisco Non-Stop Forwarding Awareness) in OSPF, EIGRP, BGP and IS-IS.

In HA deployments, NSF provides forwarding for L3, and SSO provides forwarding for L2

So NSF basically works with dynamic routing protocols, but also HSRP.

FHRP - SSO Aware HSRP:
http://www.cisco.com/en/US/docs/ios/12_2s/feature/guide/fshsrpss.html

Configuring NSF with SSO Supervisor Engine Redundancy:
http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/31sg/configuration/guide/NSFwSSO.html

By now you're probably wondering how this works together with BFD? Me too. Pete Welcher at Netcraftsmen has an excellent article putting NSF, SSO and BFD in perspective:
http://www.netcraftsmen.net/component/content/article/68-network-infrastructure/784.html

By the way, since we're on the topic of layer 2 redundancy, make sure to check out Cisco's "Switching Kitchen", pretty good stuff :-)
http://www.cisco.com/en/US/prod/switches/sk_index.html

Sunday, July 18, 2010

Book Review - Nmap Cookbook - The fat-free Guide to Network Scanning

I just read the Nmap Cookbook, and have no problem recommending it.

For security geeks, books like the new Nmap Cookbook is a real treat. It's a really good read, with a clean, structural breakdown of the different features of nmap. It also contains a really nice cheat-sheet which is great to have whenever you need a quick reference.

The book also covers the Nmap Scripting Engine (NSE), the Ndiff utility, and a nice "Tips and Tricks" section.

I didn't know that you could go beyond XMas scans and customize which TCP Flags should be set in the packets:

$ sudo nmap --scanflags SYNURG 192.168.1.40

That's pretty awesome.

All in all, a good read for the beginner and the advanced. Whatever you know about Nmap, there's probably a good chance you'll learn something new from this book.

For those of you unfamiliar with nmap, be sure to check it out. And if you want to go more in depth, Fyodor's own book comes highly recommended.

Saturday, July 17, 2010

ICMP Error Messages can be a good thing!

Too often I see this issue in CPE deployments where it generally can cause hours of troubleshooting and endless headaches. Be very careful when disabling IP Unreachables on routed interfaces ("no ip unreachables" interface-command on devices running Cisco IOS)

So what are IP Unreachables (ICMP Unreachables) and why are they so bad?

The ICMP Unreachable is one of the many ICMP Error Messages.
Straight from the source - RFC 792 - specifies the details of ICMP. Let's have a look at the 8-bit "code" field in the ICMP Destination Unreachable Message:

0 = net unreachable;
1 = host unreachable;
2 = protocol unreachable;
3 = port unreachable;
4 = fragmentation needed and DF set;
5 = source route failed.


ICMP Unreachable Type 4 is needed to calculate the PMTUD
The Path MTU is the smallest MTU of all the links in a path. PMTUD (Path MTU Discovery) enables the automatic calculation of the optimal MTU that may be used across the end-to-end communication.
So if we imagine an ICMP Destination Unreachable Message with the type-field set to 4 - "fragmentation needed and DF set", this would probably be useful in scenarios where MTU is important, right?

The point is that PMTUD uses ICMP unreachables messages to see how "high it can go" on the path MTU without requiring fragmentation (ICMPs are sent with DF-bit on, and the reply dictates how the PMTU will be calculated).
So when ICMP Unreachables are blocked with the "no ip unreachables" command, PMTUD is broken which could be a very bad thing (this is sometimes referred to as PMTUD black-holing)

But it's hip to block ICMP Unreachables!
Of course it is. But how about knowing your network instad of following "guides" and deploying templates? Unfortunatly, the many of the "secure IOS templates" out there encourages the use of "no ip unreachables" and gospels it out as best practice. Even the Cisco "AutoSecure" feature disables ICMP Unreachable on all interfaces.

The Security aspect
Security-wise, this was probably a security recommendation as an attack exploiting this could exhaust the Router CPU because these types of ICMP messages are sent to the control plane to be processed in the CPU. There was a time when the CPUs of routers weren't as powerful as they are today.

In reality, the classic attack to exploit this "vulnerability" would be the well-known "smurf attack", which could bring down the router with the evil power of ICMP knowledge.
But does the fear of the smurf still apply in todays modern infrastructure, or did this threat die out with the mullet cut in the 80's? As a DoS attack, it is not very likely that a smurf attack would bring down your network today. But if we put our tinfoil hats on, an attacker could probably use it to ennumerate hosts on our network. So we should probably consider the security aspect on some of our routers, right?

The workarounds
Luckily, your network doesn't have to be either secure or functional. It can be both. Being paranoid about ICMP Unreachables is probably a good thing when implementing border routers facing the Internet.
It may also ease us to know that Cisco has actually implemented a default ICMP rate-limiting in IOS (google it). Probably another good reason not to worry about the DoS/CPU exhaustion aspect of this issue.

Configuration-wise, some workaround exists to mitigate these issues, and this has developed into a Service Provider "best practice" on CPE devices - it is to adjust the TCP MSS. The MSS is even adjusted by default in the Cisco ASA Platform. For more details on how MSS is affected by this issue, RFC 2923 is a good place to start.

In order to explicitly allow PMTUD, but still block other types of ICMP messages, one could do "permit icmp packet-too-big" in an ACL.
Yes, this ACL would be more CPU-intensive than "no ip unreachables" applied to an interface, but we're not too worried about CPU anymore, right?

Other than that, there is also the "mls rate-limit unicast ip icmp unreachable acl-drop 0" command, which only drops unreachables when blocked by an ACL.

The Future
IPv6 alleviates these issues to a certain degree, and a newer initiative to adresse PMTUD black-holing is Packetization Layer Path MTU Discovery (PLPMTUD) as described in RFC 4821

The bottom line
So, will blocking ICMP Unreachables in your network cause the IETF to descend in black helicopters on your roof and tell you how bad it is? Probably not, but some of your applications mightstop working, so make sure you address the issue before those applications are deployed. Either that, or turn off your phone.


Recommended Further Reading:
http://www.cymru.com/Documents/icmp-messages.html
http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800d6979.shtml
http://www.nil.com/ipcorner/IP_Fragmentation/

Could it be a routing loop?

Sometimes it's good to be in the loop, sometimes not!
I'll start blogging about networking stuff, and will try to update this bad boy at least once a {day|week|month|year}! So there you have it.