This is just a short guide on how to find the main offenders in case of web server hammering.

Sample of eventual output:

Could be applied to any other sort of TCP abuse of course – just grep some other port (i.e. instead of “:80″ do “:25″ for mail for instance).

I will break it down with some basic explanations along the way for those less familiar with the command line.

First of all – how many connections are there to the web server:

netstat is a very versatile tool.
In this case, the flags being used state the following:

“-n” Numerical representation of the hosts rather than attempt to resolve to addresses
“-a” All traffic (listening and non-listening sockets)
“-t” TCP traffic only (UDP is a whole other ballgame)
“-p” The PID of the process using the port – Just a course of habit for me – since I usually want to know who is listening and taking up a port

Since this example deals with a web server, port 80 is a pretty good representation of a connection to the web server.

Count the lines (just to see what kind of numbers we are dealing with).

A typical output for netstat -natp | grep :80:

All addresses have been obfuscated to protect the innocent…

The column we are mainly interested in is the “Foreign Address” column.
Here we see the origin of the connections to our server.

A small tip that is a bit beyond the scope of this post, but is worth going off topic for a few moments – if you have an overwhelming amount of TIME_WAIT states – you might want to check the sysctl variable “net.ipv4.tcp_fin_timeout” which is by default 60 seconds, which can be quite a lot for heavy traffic web servers.

Anyhow, from here we will do a little text manipulation with the aid of *nix native tools: awk, sort and uniq – to get a nice representation of the top port 80 tcp offenders.

Will give us the fifth column:

awk of course is a very powerful tool – however here we will just be using its most common function – printing a specific column.
We still need to clean it up a bit though – since we don’t really care about the remote port.

In this case we can either invoke another awk, or use the simple tool cut – here are the two options:

These two will basically do the same thing: in awk, the “-F” flag states the field delimiter (in this case the colon “:”) and print the first column.
With cut, the “-d” flag states the delimiter (in this case the colon), and “-f1” tells it to use the first field.

Now we finally have a simple clean list of lots of IPs.

All that is left is to sort them, count how many unique IPs there are and sort the output of that test.

First we must sort, otherwise uniq doesn’t work.

“-c” tells uniq to count the occurences of each unique object.

In sort, “-n” tells it to do a proper numerical sorting rather than alphabetical, otherwise “10″ will come before “2″.

Finally the one-liner and its output:

This means that 184.106.21.219 and 216.242.75.236 alone comprise more than half of all connections to the server at port 80 – might raise a few red flags…

It may seem a bit cumbersome at first, but before you know it, one-liners like these become a second nature.

Hope you find it helpful!

Tom.