Get a continuously updating display of client IP addresses on a web server using X-Forwarded-For

Sometimes it’s desirable to have a continuously updating display of the IP addresses which are hitting a web server, with an indication of how many times each IP address has made a request. This may be because you suspect a DoS or DDoS attack, or there may appear to be some other odd activity, or you may simply be curious. If a web server is sitting directly on the Internet then it’s possible to do this fairly easily with a tool such as netstat. Often, however, a web server is behind an ELB or another type of load balancer, which means that if you try to use netstat then you’ll just see the load balancer’s IP address, not the address of the client which made the request. But if your load balancer is passing the X-Forwarded-For header (as it really ought to be) then you can use this header instead of the client IP to get a continuously updating display.

First, you’ll need to make sure the tools ngrep and watch are installed. Then copy and paste the following Bash script (replacing eth0 with the correct interface if it’s not eth0) (or grab it from GitHub):

#!/bin/bash

ngrep -il -d eth0 -W byline "x-forwarded-for" "port 80" | grep -i x-forwarded-for | 
  awk -F '[., ]' '{printf( "%s.%s.%s.%sn", $2,$3,$4,$5 );}' > /tmp/ngrep.tmp &

watch -tn 10 'cat /tmp/ngrep.tmp | sort -n | uniq -c | sort -nr | head -30'

When you run this script you should see a continuously updating display which looks something like this:

80 11.22.33.44
60 55.66.77.88
40 44.33.22.11
20 88.77.66.55
...

It may start to slow down after a while. If that happens then just hit CTRL-C and run it again.

Python script to convert ICS-exported List from Reminders app to plaintext

I make extensive use of the Reminders app* in OS X to keep track of tasks and to-do items, and I wanted a way to export a list of reminders to plaintext, so I knocked up a quick Python script to take an ICS file exported from a List in Reminders (which you can do from the File menu) and output it in plaintext. If this is something you find yourself needing to do then this might work for you.

Grab the script from GitHub or copy and paste it below:

#!/usr/local/bin/python
# -*- coding: utf-8 -*-

import re
from optparse import OptionParser

parser = OptionParser()
parser.add_option("-f", "--file", dest="filename", metavar="FILE", 
  help="read reminders from ICS FILE")
parser.add_option("-c", "--include-completed", action="store_true", 
  dest="include_completed", help="include completed tasks")
(options, args) = parser.parse_args()

if not options.filename:
  print "No filename specified; -h for help"
  exit(1)

with open(options.filename) as file:
  ics_export = file.readlines()
file.close

todos = []
for ics_line in ics_export:
  if "STATUS:COMPLETED" in ics_line and options.include_completed is None:
    exclude = True
  elif "STATUS:" in ics_line:
    exclude = False
  if "SUMMARY:" in ics_line and exclude is False:
    bits = re.split(":", ics_line)
    todos.append(bits[1].rstrip())

todos.sort(key=lambda y: y.lower())
for todo in todos:
  print "•", todo

* Synced to my iPhone and other Macs via ownCloud on my Linux server, but that’s a story for another day.