[CLUE-Tech] HP ethernet switch UDP broadcast storm

[Jim Ockers]

Hi Chris,

Thanks for your reply and suggestions.

> When you say "mesh" are you referring to redundant links between switches?
> If so, I'm assuming you're using spanning tree to remove loops.

I meant to include this information and forgot.

We have no redundant loops that we know of.  Spanning tree is turned
off for all our switches.  All of the switches are directly "backhauled"
to switch2.  The "mesh" is HP's word for the redundant links, but we
have it set up so that switch2 is a massive SPOF.

It's simpler that way even if less reliable.  In 3 years we've had only
two massive switch failures and we have spare parts on hand.  One time
the supervisor card in one of the edge switches died, that was fun.

> Even though it was a layer 3 broadcast it got treated as a layer 2
> broadcast because the switches won't have an ARP entry for a broadcast
> address. At least, they shouldn't. :) So that would explain why you saw a
> SINGLE copy of the particular packet all over, but doesn't explain why you
> continued to see it flooded all over.
> 
> Was anyone in your closet plugging stuff in at the time this happened? Any
> possibility that someone created another loop in the network? Was anyone
> plugging in switches anywhere else in the network?

No, nothing like that was going on that we know of.  We have had the
occasional person connect 2 ports of a little linksys switch back to
two ports of the core using crossover cables, which creates a problem, 
but we've never seen this type of problem before.

> Sounds like there was a loop someplace or a new switch with a higher
> priority got plugged in and spanning tree got confused. This gains
> credence especially when you say the problem got fixed by disconnecting
> "mesh" ports, which will potentially physically remove an existing loop,
> and also cause spanning tree to recalculate.

We thought about the possibility of a loop but we decided that couldn't
be the problem because we have spanning tree turned off.

===========================- TELNET - MANAGER MODE -============================
                 Switch Configuration - Spanning Tree Operation

  Spanning Tree Enabled [No] : No
  STP Priority [32768] : 32768          Hello Time [2] : 2
  Max Age [20] : 20                     Forward Delay [15] : 15

> Hope that helps... hard to tell exactly what happened. I've seen spanning
> tree fail before, particularly if someone is plugging cables in and out
> faster than spanning tree can keep up.
> 
> Even if someone plugs in a switch in their office it has the potential to
> cause problems if that shiny new switch has a higher priority - it will
> trigger the tree to recalculate.
> 
> Fortunately this doesn't happen very often.

Thanks,
Jim

-- 
Jim Ockers, P.Eng. (ockers at ockers.net)
Contact info: please see http://www.ockers.net/