Using Linux as a Small Business Internet Gateway, Part 2
Pages: 1, 2, 3
Monitoring Traffic Balance
Monitoring traffic balance is really only two tasks: taking care of speed of loading of our channel and counting all traffic for a period of time. For the first task we can use mrtg, conveniently included in the distribution.
Before installation, you'll also have to install SNMP support.
# rpm -ihv net-snmp-5.0.6-17.i386.rpm mrtg-2.9.17-13.i386.rpm
Preparing... ########################################### [100%]
1:net-snmp ########################################### [ 50%]
3:mrtg ########################################### [100%] Let's first configure snmp. We need to modify file /etc/snmp/snmpd.conf. Add the following lines to the Access Control section:
# Minimal setup for MRTG at localhost:
com2sec local 127.0.0.1 system
group ROView v1 local
group ROView v2c local
view roview included .1.3.6.1.2.1
access ROView "" any noauth exact roview none noneThen start snmp:
# service snmpd start
Starting snmpd: [ OK ]We can now check how snmp works:
$ snmpwalk localhost publicIf everything is all right, the results will be something like this:
system.sysDescr.0 = Linux localhost.localdomain 2.4.20-8asp #1
Thu Mar 13 20:16:12 EEST 2002 i686
system.sysObjectID.0 = OID: enterprises.ucdavis.ucdSnmpAgent.linux
system.sysUpTime.0 = Timeticks: (10408) 0:01:44.08
system.sysContact.0 = Root <root@localhost>
(configure /etc/snmp/snmp.local.conf)
system.sysName.0 = localhost.localdomain
system.sysLocation.0 = Unknown (edit /etc/snmp/snmpd.conf)
system.sysORLastChange.0 = Timeticks: (30) 0:00:00.30
system.sysORTable.sysOREntry.sysORID.1 = OID: ifMIB Now we need to configure mrtg. Its package includes a utility to create configuration files. Connect to the Internet and issue the next command:
# cfgmaker --global 'WorkDir: /var/www/html/mrtg' \
--global 'Options[_]: growright, unknaszero, bits' \
--global 'Language: russian' \
--global 'AddHead[_]: <META http-equiv="Content-Type"
content="text/html; charset=koi8-r">' \
--output '/etc/mrtg/mrtg.cfg' \
system@localhost This creates the /etc/mrtg/mrtg.cfg file. We need to change it a bit, because the speed of the PPP interface (ppp0, ppp1, etc.) cannot be correctly determined. The generator just comments out these lines. We need to uncomment the PPP section to set the maximal speed (argument MaxBytes) correctly. As a result, we should get something like:
### Interface 2 >>: 'ppp0' | Name: '' | Ip: '123.4.5.6' | Eth: '' ###
### The following interface is commented out because:
### * has a speed of 0 which makes no sense
#
Target[localhost_2]: 2:system@localhost:
SetEnv[localhost_2]: MRTG_INT_IP="123.4.5.6" MRTG_INT_DESCR="ppp0"
MaxBytes[localhost_2]: 115200
Title[localhost_2]: Traffic Analysis for dialup -- localhost.localdomain
PageTop[localhost_2]: <H1>Traffic Analysis for dialup -- localhost.localdomain</H1>
<TABLE>
<TR><TD>System:</TD> <TD>localhost.localdomain in Unknown </TD></TR>
<TR><TD>Maintainer:</TD> <TD>Root <root@localhost></TD></TR>
<TR><TD>Description:</TD><TD>ppp0 </TD></TR>
<TR><TD>ifType:</TD> <TD>ppp (23)</TD></TR>
<TR><TD>ifName:</TD> <TD></TD></TR>
<TR><TD>Max Speed:</TD> <TD>115.2 KBytes/s</TD></TR>
<TR><TD>Ip:</TD> <TD>123.4.5.6 (123.dialup.provider.net)</TD></TR>
</TABLE>Everything is almost ready for the first start. First, move the contents of /var/www/html/mrtg into /usr/share/doc/mrtg-2.9.17/doc. This is the documentation, for some reason placed into a web directory.
Let's start mrtg manually:
# /usr/bin/mrtg /etc/mrtg/mrtg.cfg
Rateup WARNING: /usr/bin/rateup could not read the primary log file
for localhost_2
Rateup WARNING: /usr/bin/rateup The backup log file for localhost_2 was invalid
as well
Rateup WARNING: /usr/bin/rateup Can't remove localhost_2.old updating log file
Rateup WARNING: /usr/bin/rateup Can't rename localhost_2.log to localhost_2.old
updating log file
# /usr/bin/mrtg /etc/mrtg/mrtg.cfg
Rateup WARNING: /usr/bin/rateup Can't remove localhost_2.old updating log file All error messages during the first two starts are normal — all data files and backup files must be created. You will not get these errors again. Reports will be generated in the /var/www/html/mrtg directory, with a separate file for each interface, following the pattern localhost_<interface_number>.html. During analysis, you can easily create a page of cumulative information about all interfaces. You can also turn on automatic report updating. In /etc/cron.d/mrtg, uncomment the line:
0-59/5 * * * * root /usr/bin/mrtg /etc/mrtg/mrtg.cfgand append the path where we will keep the log:
--logging /var/log/mrtg.logTruthfully, you can even skip this step. If so, any errors will just be emailed to the administrator.
Now it's time to solve the traffic counting problem. Honestly, I was unable to find any easy-to-configure package that could count all of the things I wanted to count. For example, my provider counts my incoming traffic from foreign resources, but traffic from the provider's resources shouldn't be counted.
As I discovered, there is no such problem. Everything can be easily solved using just iptables and little calculation utility. In theory, iptables can count traffic; we just need to create rules for counting. Pay close attention to the rule changes here:
# Simple firewall for internet gateway
*filter
:INPUT ACCEPT [0:0]
# Deny forward for security reasons
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:RH-FW-INPUT - [0:0]
:trafin - [0:0]
:trafout - [0:0]
:trinloc - [0:0]
:troutloc - [0:0]
# special counter rules
-A trafin -j RETURN
-A trafout -j RETURN
-A trinloc -j RETURN
-A troutloc -j RETURN
# Redirect all input packets to our special rule
-A INPUT -i ppp0 -j trafin
-A OUTPUT -o ppp0 -j trafout
-A INPUT -s 123.4.0.0/16 -i ppp0 -j trinloc
-A OUTPUT -d 123.4.0.0/16 -o ppp0 -j troutloc
-A INPUT -j RH-FW-INPUT
# Accept all packets for local interface (lo)
-A RH-FW-INPUT -i lo -j ACCEPT
# Define custom rules
# Access to mail services
-A RH-FW-INPUT -p tcp -m tcp --dport 25 -j ACCEPT
-A RH-FW-INPUT -s 192.168.0.0/24 -p tcp -m tcp --dport 110 -i eth0 -j ACCEPT
-A RH-FW-INPUT -s 192.168.0.0/24 -p tcp -m tcp --dport 143 -i eth0 -j ACCEPT
# Access to DNS service
-A RH-FW-INPUT -s 192.168.0.0/24 -p tcp -m tcp --dport 53 -i eth0 -j ACCEPT
-A RH-FW-INPUT -s 192.168.0.0/24 -p udp -m udp --dport 53 -i eth0 -j ACCEPT
# Access to HTTP service
-A RH-FW-INPUT -s 192.168.0.0/24 -p tcp -m tcp --dport 80 -i eth0 -j ACCEPT
# This part equal to some part from generated by Lokkit utility
-A RH-FW-INPUT -p tcp -m tcp --dport 143 -j ACCEPT
-A RH-FW-INPUT -p tcp -m tcp --dport 0:1023 --syn -j REJECT
-A RH-FW-INPUT -p tcp -m tcp --dport 2049 --syn -j REJECT
-A RH-FW-INPUT -p udp -m udp --dport 0:1023 -j REJECT
-A RH-FW-INPUT -p udp -m udp --dport 2049 -j REJECT
-A RH-FW-INPUT -p tcp -m tcp --dport 6000:6009 --syn -j REJECT
-A RH-FW-INPUT -p tcp -m tcp --dport 7100 --syn -j REJECT
# Allow access to external mail/news/DNS
-A FORWARD -s 192.168.0.0/24 -p tcp -m tcp --dport 25 -j ACCEPT
-A FORWARD -s 192.168.0.0/24 -p tcp -m tcp --dport 110 -j ACCEPT
-A FORWARD -s 192.168.0.0/24 -p tcp -m tcp --dport 143 -j ACCEPT
-A FORWARD -s 192.168.0.0/24 -p tcp -m tcp --dport 119 -j ACCEPT
-A FORWARD -s 192.168.0.0/24 -p tcp -m tcp --dport 53 -j ACCEPT
-A FORWARD -s 192.168.0.0/24 -p udp -m udp --dport 53 -j ACCEPT
COMMIT
# Enable masquarading
*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A POSTROUTING -p tcp -s 192.168.0.0/24 -o ppp0 -j SNAT
--to-source 123.123.123.123
-A POSTROUTING -p udp -s 192.168.0.0/24 -o ppp0 -j SNAT
--to-source 123.123.123.123
COMMITNow, after reloading iptables, you can easily see all traffic size:
# iptables -L trafin -vxn
Chain trafin (1 references)
pkts bytes target prot opt in out source destination
44 3008 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
# iptables -L trafout -vxn
Chain trafout (1 references)
pkts bytes target prot opt in out source destination
44 5008 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0and for local traffic:
# ipchains -L trinloc
Chain trinloc (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
# iptables -L troutloc -vxn
Chain troutloc (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0Please note that when you reload iptables, all counters will reset. Besides, it's useful to see daily, weekly, and monthly traffic balances in a good visible presentation.
That's the purpose of the attached iptrafcount.pl program. This program accumulates data in /var/log. You can receive a short help message by running it with the -h argument. To use it, you should also do the following:
Copy the program itself into /usr/sbin and make it executable:
# chmod +x /usr/sbin/iptrafcount.plAdd the following line to the /etc/crontab line:
*/5 * * * * root /usr/sbin/iptrafcount.pl -l
That's all. Soon, you will be able to check reports (for example, for March 2003):
# iptrafcount.pl -w common -d 03/2003The program outputs HTML. You can easily include it in your custom CGI script.
Conclusion
The two parts of this article have explained how to use Linux as an Internet gateway server. Having installed several similar installations, I'm confident that Linux does its job perfectly well. Furthermore, in many cases, I personally economize my time and the time of my clients by working from home without having to come into the office.
Over time, any organization or company interested in diving more deeply into the Internet can easily expand its gateway into a fully functioning web or DNS server.
Alexander Prohorenko is a certified professional, who holds Sun Certified System Administrator and Sun Certified Java Programmer certifications.
Return to the LinuxDevCenter.com.
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 2 of 2.
-
a few tidbits...
2003-12-29 08:40:33 anonymous2 [Reply | View]
(background: i set up "Small Business Internet Gateways" for a living. right now i have about 20 clients, ranging from mom-n-pop size businesses to 25 person design engineering companies.)
regarding your two part article, it's a pretty good compendium, but i have a few comments:
a) the average small business doesn't need or want squid. there is very little temporal or spatial locality to the objects that "internal" surfers are accessing. KISS -- keep it simple and stupid, unless you like fixing stuff 24x7. squid is a memory, disk, and CPU hog -- not what you need on your gateway.
b) in part 1 you recommend against sendmail as being "too complicated". then in part 2 you go ahead and install BIND? google "djbdns" and get BIND out of the picture. i too used to wrestle with BIND, spending hours with cricket's book in my lap. then i discovered djbdns and now i wish i could get back all that time i spent with BIND. for your average small business with an internal domain, djbdns is sooooo much easier to install and configure. it works, it's secure, adding new records is trivial, end of story.
c) work in stages and with security in mind. configure the firewall first, default DENY, and then start adding services. install squid before setting up the firewall? that's the wrong order. install tripwire (or one of it's equivalents) so you can see when you get rooted. :^)
d) the first step should be to delete all the RPMs of stuff you DON'T need. this will 1) make updating easier, 2) improve security, and 3) prevent people from mucking with things they don't understand. (case in point... clients who email me and say "i noticed that i can't seem to run a graphical desktop on the gateway console, and i want to reconfigure the firewall to allow my son to IM with his friends from my office desk.")
e) backups. backups. backups. you are insane if you think that a "small business" is going to accept a few days of downtime while you rebuild their server and website from scratch. instead of installing squid as the first task (totally unneeded, as noted above), install a backup system that will allow you to recover the ENTIRE box in 8 hours or less. disks fail, plan for it or you'll have unhappy clients. for this reason i suggest coming up with a standard config that fits 90% of your clients needs. then, the last 10% you have to work with, but keeping commonality across boxen makes your like a lot easier.
f) monitoring traffic looks sexy to the client. but totally extraneous for you. instead, monitor CPU utilization, disk space, and process creation rate. these are the things that ultimately get you in trouble. example: it should not be a surprise when the gateway runs out of disk space in 10 months, 'cause /var/spool/mail is full of crap. use MRTG to plot disk usage from day one. after a month or two, it will not be difficult to extrapolate when the disks are going to max out. and this can be a GREAT early warning indicator that your ftpd config has been compromised -- all of sudden disk usage soars as the latest Matrix movie mpegs are hosted on your box. another example: notice that /usr is growing? find the log file that is located on /usr and move it NOW. process creation rate is another great indicator that something is amiss. when a busy box goes from 1-5 per sec to 50 per sec it's time to look into what the problem is.
g) externals: have a backup mail server (MX) record configured for your client, so if their server takes a dump their email (which = MONEY) continues to get deposited somewhere that they can get to it. and, have an external DNS record configured so that you don't have to do zone transfers across the client gateway.
hope all that leads to a more disciplined approach.
This document have been really helpful to do traffic monitoring.
Best regards.