I had no idea my blog was so popular. 100,000 page views in ten days. That’s 10,000 views a day! What kind of crack am I selling? A quick check of the logs indicates it is merely referrer spam. I’ve never seen it so bad, though. Serious brute force is in play. Time to clean up.
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule ^$ %{HTTP_REFERER} [R,L,E=nolog:1]
Most of them had no UA. That probably won’t last forever. A much larger site would probably have all this lost as noise anyway.
Finally, a fix for corrupt logfiles. I am anal about logs.
for file in $(seq 22 2) ; do
zcat /var/loggies/edseek.com_access_log.$file.gz |\
perl -ne 'if($_ !~ /^.*"([^"]+)"\s"-"$/ || $1 =~ /edseek/) { print; }' > /tmp/edseek_real.log
awstats.pl -config=edseek -update -LogFile=/tmp/edseek_real.log"
done
Another option would be to discard log entries where the IP only hit a single page within some period of time. As stats packages adopt nofollow, I hope this becomes less of an issue. Like, email spam, it’s a perpetual arms race you can’t win.