Ones and Zeros

First, happy square root day!  And do you have your plans set for pi day?

At work we had a problem with accessing the internet today.  Our IT person - a sharp guy - had found that it wasn’t an outside problem, but one on our own internal network.  His conclusion is that had two devices on one IP address.  Unfortunately that address was the main gateway address so some of the packets that should leave our network for the internet were not doing so.

We’re a small company, we don’t really have any network monitoring.  And our “smart switches” are well, short a few chips from “smart”.  So the only way to track that device down is the painful step of unplugging each wire from the main switches, testing, the trying the next one.  Doing a binary search is better, but still painful.

But I worked out a way to find the problem device.  I used ARP and the ARP cache along with the “ping” command to find it.  First, some background.

When you use the network, even for a simple ping command, your computer first has to find where to send those data packets.  On ethernet-based networks the real network address is an ethernet address, each then has an IP address assigned to it as well.  At the actual network level though, all the data packets are addressed with ethernet addresses.

But the higher level software (your email or web browser for example) wants to send packets to an IP (Internet Protocol) address.  It’s called an internet protocol since it connects multiple networks.  That “www.google.com” URL is translated to an IP address to be used in your browser software.

So how does your computer knows which ethernet address to use when all the software has are IP addresses?  The ARP - address resolution protocol - does the trick.  Your computer sends out an ARP broadcast packet that says “who is assigned the IP address xx.yy.zz.aa?”.  The reply is from the device that has that IP address and your computer uses that reply’s ethernet address to send the packet to.  Of course this address may be the gateway from your ethernet to another network where it would eventually get forwarded to google in my above example.

It hit me I could use ethernet and ARP to figure out the problem device.  I cleared the ARP cache in my machine, and pinged the problem IP address we had.  This forced that ARP broadcast.  Then I dumped the ARP cache and found the ethernet address of the problem machine.  I looked up the vendor code (the first three bytes of the ethernet address) in the IEEE listing.  We run a mixed shop and fortunately there weren’t many machines of this type.  I was able to examine a short list of machines of that type and found the problem machine.

Oh, and the actual problem?  An inadvertent misconfiguration of a proxy on that system duplicated the gateway’s IP address.  Nicely solved once we found that figured out that a proxy was on that system.

This entry was posted on Tuesday, March 3rd, 2009 at 21:22 and is filed under software, technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.