Bandwidth monitoring with iptables (Linux.com)
Most of the time we use iptables to set up a firewall on a machine, but iptables also provides packet and byte counters. Every time an iptables rule is matched by incoming or outgoing data streams, the software tracks the number of packets and the amount of data that passes through the rules."
Posted Dec 27, 2005 2:40 UTC (Tue)
by hisdad (subscriber, #5375)
[Link] (3 responses)
--Dad
Posted Dec 27, 2005 22:06 UTC (Tue)
by zblaxell (subscriber, #26385)
[Link] (2 responses)
Per-connection (i.e. TCP connections and UDP peer associations) is possible with some of the netfilter patch-o-matic patches. I'm not sure how useful they are given they don't count a lot of cases (TCP stealth pings, broadcasts, ...) and IIRC they don't report statistics to a log until the connection terminates (which in the case of a TCP connection with a dead peer is a very long time).
Per-connection (i.e. VPN tunnels, PPPoE, and other dynamic interfaces) is usually possible with help from whatever is managing the connections, or from a script called by the same (read the packet counters on the interface when it comes up and goes down). I've used the interface naming feature ('ip name ...') to force individual connection sessions to match a pool of named iptables rules--not the best way to do this kind of accounting, but certainly possible.
iptables does provide atomic-read-and-zero on the packet counts, so it's certainly not sloppy. I've used it to check the accuracy of my ISP's billing systems (and found a bunch of bugs in every one).
Iptables is limited if you want anything other than a few dozen pre-defined aggregate counts, so it's not suitable at all for exercises in discovery like "which are the most popular subnets accessing my network?" Grepping webserver logs or running an observing monitor like iftop, iptraf, tcpdump-and-a-Perl-script, etc. is a better tool to answer that kind of question.
Posted Dec 28, 2005 7:38 UTC (Wed)
by hisdad (subscriber, #5375)
[Link] (1 responses)
That was an interesting excerise in itself, as the IPtables dump gives a
It is certainly sloppy in that, because its not stateful, you have to make reasonable guesses as to what the send and reply traffic is like going forwards and not get it tangled with send and reply traffic going backwards.
I'm loath to try out of kernel patches, for reason too obvious to bear discussion.
I hope that in a few years the patches will go mainstream.
Until then, I'll stick with usefull, but not perfect.
--Dad
Posted Jan 1, 2006 6:25 UTC (Sun)
by zblaxell (subscriber, #26385)
[Link]
iptables -t mangle -I PREROUTING -i eth0 -m mark ! --mark 0xfee1dead
The results look like this (iptables -t mangle -xvnL):
You simply extract the lines you are looking for by grepping for the appropriate "fee1dea" string, e.g.
incomingEth0=$(iptables -xvnL -t mangle | awk '/fee1dead/ { print $2 }')
Using the PREROUTING and POSTROUTING chains means you don't have to worry about which packets are INPUT and which are FORWARD, although since the mangle table happens before the filter table you will have to worry about packets that were received or proposed for transmission, but later dropped.
I don't really understand what you mean by "send and reply traffic is like going forwards and not get it tangled with send and reply traffic going backwards". Are you trying to count traffic on connections that are initiated on different sides of the gateway in separate groups? CONNMARK can do that, and AFAIK CONNMARK is part of the standard kernel (the only extension I use these days above what comes in 2.6.x. is ipt_random). CONNMARK allows you to copy marks from packets to connections, so if you have:
1. one set of iptables rules that marks connections based on which interface they were initiated on,
you get something that sounds to me like what you want...but I'm not sure. I use CONNMARK mostly for dynamic load balancing and failover between interfaces connected to two different ISP's, but it's quite versatile.
Yeah, its simple in theory.Statelessness a great pain
Then you realise that you need to match forward and reverse packets.
And you can't on a connection basis.
Then things get horribly messy and the results are sloppy.
www.shorewall.net has a good tutorial on IP accounting.
The best you can say is that it is ok for most purposes.
If you're serious about IP accounting using iptables, you're going to end up polling the iptables counters every $(interval that is short enough that you don't care if its data is lost), and feeding that data into persistent storage, either through iptables save and restore, or feeding the data into a database--the latter leaves an audit trail which helps filter out bogus data or fill in lost data from other sources. Simply "reading the counters at the end of the month" won't work if you upgrade the kernel, or have a power failure, or change the rule sets, etc.Statelessness a great pain
With respect to database reading, I am using acounting chains set up in shorewall and reading them every 300 secs into RRDTool.Statelessness a great pain
wordy, textual output. I tried using sed and grep without success and ended up using command line PHP, to get the numbers and put them into rrdtool.
Hmmm...I'm not sure how shorewall structures things, but in iptables it should be easy: create rules in the mangle table in the PREROUTING and POSTROUTING chains, which test for the absence of marks with magic values, e.g.:Statelessness a great pain
iptables -t mangle -I POSTROUTING -o eth0 -m mark ! --mark 0xfee1deae
iptables -t mangle -I PREROUTING -i eth1 -m mark ! --mark 0xfee1deaf
etc.
Chain PREROUTING (policy ACCEPT 58525851 packets, 39026491526 bytes)
pkts bytes target prot opt in out source destination
189 18542 all -- eth0 * 0.0.0.0/0 0.0.0.0/0 MARK match !0xfee1dead
incomingEth1=$(iptables -xvnL -t mangle | awk '/fee1deaf/ { print $2 }')
2. another set of iptables rules that copies marks from connections onto unmarked packets, and
3. another set of iptables rules that count the various different mark values