[clue] [tech] why would iptables not masquerade some IMAP/S (993/tcp) packets

[Jim Ockers]

Hi CLUEbies,

As usual when I can't figure something out I like to ask the list.  I 
have a Linux 2.6.32.27 (armv5teb - Intel XScale) system which acts as a 
router and has a variety of network connections including 3G cellular.  
The linux firewall is configured to masquerade internal network traffic 
so that it should appear to come from the router's own outside 
interface.  Unfortunately for some reason some packets aren't NATted and 
manage to escape through the PPP interface still bearing the (invalid) 
internal IP address.

When I use a 3G PPP interface as the default gateway, and the 3G data 
provider happens to be Verizon, then Verizon's network security (which 
has a threshold of invalid packets/time) disconnects the PPP connection 
by sending an LCP Termination Request.  Normally this would not be a 
problem but somehow this router/firewall are letting some packets sneak 
through without NATting them.  I attached a screen shot of the offending 
packets, which are from a PPP record file created by the "record 
/tmp/filename" option and which wireshark knows how to decode.  The 
recorded file contains all bits written to or received from the serial 
port during the course of the PPP daemon running, and so there should 
never be any non-NATted packets show up in this file.  The screen shot 
shows a few and this was enough to get Verizon to disconnect.

Since I can't attach files and send them to the list here is the screen 
shot: http://chile.mudlogsys.com/~ockers/wireshark.png
<IMG SRC="http://chile.mudlogsys.com/~ockers/wireshark.png">

The relevant fine print & details:
 - The default route is the PPP interface device, no gateway specified
 - There is no iproute2 or other routing funny business
 - Some of the other iptables prerouting/etc rules are a bit wonky but 
the nat/POSTROUTING and mangle/POSTROUTING are quite clear - see below
 - The raw table has nothing in it
 - We ARE doing traffic shaping using u32 classifier and imq0.  All 
packets are looped through the IMQ device for htb prioritization.
 - The source IP address in the packets as shown in the screen shot is a 
garden variety Apple macbook laptop.
 - Note that every one of the un-masqueraded packets in this capture are 
TLSv1 to port 993 (IMAP/S).  I think this is simply a co-incidence 
because I've seen other types of traffic in other disconnect tests 
including plain old HTTP.
 - I have a second screen shot showing all 993/tcp packets.  3 of the 16 
packets are properly NATted.  The rest are not NATted.
 - AT&T just discards improperly addressed packets.  Verizon hangs up 
the connection.  I can see that it might be nice to keep "bad" traffic 
off their network so I can't object to this policy.

http://chile.mudlogsys.com/~ockers/wireshark-imaps.png
<IMG SRC="http://chile.mudlogsys.com/~ockers/wireshark-imaps.png">

root at Router:~# iptables -t nat -L POSTROUTING -n -v
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               
destination        
    0     0 ACCEPT     all  --  *      *       10.0.0.0/8           
10.0.0.0/8         
    0     0 ACCEPT     all  --  *      *       192.168.0.0/16       
192.168.0.0/16     
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0            
0.0.0.0/0          
root at Router:~# iptables -t mangle -L POSTROUTING -n -v
Chain POSTROUTING (policy ACCEPT 488K packets, 353M bytes)
 pkts bytes target     prot opt in     out     source               
destination        
root at Router:~#

The counters are zeroed out each time it reconnects, so don't mind the 
zero pkt/byte counters there.

I don't really know what else to check for.  Given that catch-all 
MASQUERADE rule, how could any of these packets sneak past it without 
getting masqueraded?

Thanks for any advice or suggestions,
Jim

-- 
Jim Ockers, P.Eng. (ockers at ockers.net)
Contact info: http://www.ockers.net/msi.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://cluedenver.org/pipermail/clue/attachments/20110712/aafe1437/attachment-0001.html