[clue-tech] forwarded packets not matching any iptables rules?

Thu Aug 24 12:58:30 MDT 2006

Hi,

I have a very unusual iptables problem.  It seems that packets
are coming in one interface and getting copied to another inter-
face (forwarded) but are not matching any iptables rules.  Of
course this is breaking my device because the expected SNAT 
(or MASQUERADE) is not happening.

System is Linux 2.4.22, 256MB RAM (tons of free RAM), load aver-
ages are all 0.00, /proc/net/ip_conntrack has only 24 entries,
no kernel/hardware error messages of any sort shown in dmesg or 
/var/log/messages, and iptables & NIC drivers all compiled into 
kernel.  Rebooting does not help - it still does the same thing
after rebooting.

The inside interface is eth1 and outside interface is eth0.  The
default gateway is on eth0.  The packets come in eth1, are processed
by the linux kernel, and go out eth0, supposedly after getting
SNATted.

The problem is that certain packets (destined to 1194/udp on a
server of ours) are not matching any of the iptables rules.  As
I understand it the packet flow for forwarded packets is as
follows:

mangle/PREROUTING; nat/PREROUTING; filter/FORWARD; nat/POSTROUTING

I've put counters on all of those and the packets I'm interested
in are not being processed by any of those tables.  It doesn't
matter if I use SNAT or MASQUERADE because the packets never enter
the tables.  Of course /proc/net/ipv4/ip_forward is 1.

As a practical matter we have about a thousand of these systems and
we've never seen this behavior before.  Shouldn't it be impossible?
How can a packet be routed by the linux kernel but not go through
netfilter?  Isn't it impossible to bypass iptables/netfilter?

By the way lots of other packets are properly being SNATted on this
system.  It seems only these are not getting netfiltered...

Thanks for any ideas/comments or suggestions.  What can I do to make
this start working?

Thanks,
Jim

[root at Sataimer root]# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:08:9B:A9:E6:5A
          inet addr:192.168.11.10  Bcast:192.168.11.255  Mask:255.255.255.0
[root at Sataimer root]# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:08:9B:A9:E6:5B
          inet addr:172.21.165.146  Bcast:172.21.165.159  Mask:255.255.255.240

[root at Sataimer root]# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
172.21.165.144  0.0.0.0         255.255.255.240 U         0 0          0 eth0
192.168.11.0    0.0.0.0         255.255.255.0   U         0 0          0 eth1
10.0.0.0        192.168.11.1    255.0.0.0       UG        0 0          0 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
0.0.0.0         172.21.165.145  0.0.0.0         UG        0 0          0 eth0

[root at Sataimer root]# iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 44051 packets, 4360K bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            208.38.1.19

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 SNAT       all  --  *      *       0.0.0.0/0            208.38.1.19        
to:172.21.165.146
27406 1606K SNAT       all  --  *      eth0    0.0.0.0/0            0.0.0.0/0          
to:172.21.165.146

Chain OUTPUT (policy ACCEPT 169 packets, 40261 bytes)
 pkts bytes target     prot opt in     out     source               destination

The filter/FORWARD is ACCEPT.  I put tracking lines in filter/FORWARD
to try to count these packets but the counters are all zero on this
system.  It seems they are passing through the networking code without
going through netfilter?

I captured some tcpdumps to compare this system to another which is
working OK for the exact same traffic.  Note that the one that's OK
shows a different source IP address for the packets captured on eth0.
The one that is NOTOK has the same source IP address for the packets
on both eth0 and eth1.

It's supposed to do a nat/POSTROUTING SNAT in between eth1 and eth0.

[root at Sataimer-OK tmp]#  ./tcpdump -n -v -i eth1 host 208.38.1.19
tcpdump: listening on eth1
11:12:19.632492 192.168.11.100.1026 > 208.38.1.19.1194:  udp 141 (DF) (ttl 64, id 0, len 169)
11:12:20.080888 208.38.1.19.1194 > 192.168.11.100.1026:  udp 77 (DF) (ttl 52, id 17684, len 105)
[root at Sataimer-OK tmp]#  ./tcpdump -n -v -i eth0 host 208.38.1.19
tcpdump: listening on eth0
11:12:57.410824 172.21.147.242.1026 > 208.38.1.19.1194:  udp 141 (DF) (ttl 63, id 0, len 169)
11:12:57.420035 172.21.147.242.1026 > 208.38.1.19.1194:  udp 221 (DF) (ttl 63, id 0, len 249)
11:12:58.191252 208.38.1.19.1194 > 172.21.147.242.1026:  udp 77 (DF) (ttl 53, id 17736, len 105)

Sataimer, not OK: 

[root at Sataimer-notOK tmp]# ./tcpdump -n -v -i eth1 host 208.38.1.19
tcpdump: listening on eth1
11:29:44.225971 192.168.11.100.1024 > 208.38.1.19.1194:  [udp sum ok] udp 42 (DF) (ttl 64, id 0, len 70)
11:29:46.745927 192.168.11.100.1024 > 208.38.1.19.1194:  [udp sum ok] udp 42 (DF) (ttl 64, id 0, len 70)
[root at Sataimer-notOK tmp]# ./tcpdump -n -v -i eth0 host 208.38.1.19
tcpdump: listening on eth0
11:30:19.925395 192.168.11.100.1024 > 208.38.1.19.1194:  [udp sum ok] udp 42 (DF) (ttl 63, id 0, len 70)
11:30:23.246245 192.168.11.100.1024 > 208.38.1.19.1194:  [udp sum ok] udp 42 (DF) (ttl 63, id 0, len 70)

On eth0 the source IP address should be 172.21.165.146 if the SNAT
happened.

Phooey.

-- 
Jim Ockers, P.Eng. (ockers at ockers.net)
Contact info: please see http://www.ockers.net/