piston saves!

8/16/2007 – 5:03 pm

I download a prerelease of the latest Engines earlier in the summer with piston. My tests were failing in strange and magical ways. I spent quite a bit of time trying different things, then finally resorted to a fresh Rails app that I slowly populated with relevant files. Quickly I discovered the problem was my Engines plugin. I really thought I had the full release, but clearly not.

I was really relieved when I retargeted piston at the actual release version of Engines and updated. A single file change and I was back in business.

jasonb@faith:~/src$ piston switch http://svn.rails-engines.org/plugins/engines \
  vendor/plugins/engines/
Processing 'vendor/plugins/engines/'...
  Fetching remote repository's latest revision and UUID
  Restoring remote repository to known state at r600
  Updating remote repository to http://svn.rails-engines.org/plugins/engines@611
  Processing adds/deletes
  Removing temporary files / folders
  Updating Piston properties
  Updated to r611 (4 changes)

As it happened, the problem was:

+++ vendor/plugins/engines/lib/engines/testing.rb
@@ -2,7 +2,6 @@
 # for more details.

 require 'test/unit'
+require 'test_help'

Sigh. It does help to import the correct branch of the code, yes?

Because I used piston, it was stupid easy to retarget the correct version of Engines, then update and see what it was I screwed up. Further, any local changes I made -- I had none for this plugin -- would be preserved and subject to the usual Subversion conflict resolution process (such as it is).

It's best not to modify plugins lest you find yourself maintaining even more code, but if you must, at least your changes won't be clobbered. And you don't have to run diff between plugin checkouts to try to deduce which of your changes are about to get lost.

By jasonb | Posted in Rails | Comments (0)

mootools and form field hints

8/11/2007 – 1:38 am

Usually I love mootools, really. Usually. I found CSS Guy's form field hints posting inspiring, so I modified it to work with mootools. The result is something simple and easy.

$ES('div.hint').each(function(el) {
	new Element('div').addClass('hint-pointer').setHTML(' ').injectInside(el);
});
$each(document.forms, function(form) {
	$each($(form).getElementsBySelector('input, select'), function(input) {
		if($(input).getParent().getElementsByTagName('div')[0]) {
		$(input).addEvent('focus', function(e) {
			hint = $E('.hint', this.getParent());
			hint.setStyle('display', 'inline');
			hint.effect('opacity', {duration: 500, wait: false}).set(0).start(1);
		}.bind(input));
		$(input).addEvent('blur', function(e) {
			hint = $E('.hint', this.getParent());
			hint.effect('opacity', {duration: 500, wait: false}).set(1).start(0);
		}.bind(input));
		}
	});
});

I modified the CSS to suit my layouts better. You'll notice above I am using divs instead of spans. I noticed IE6 would never hide the hints div ever again once it was diplayed. Fortunately the fade effect hides it for me.

.hint {
	display: none;
	position: absolute;
	right: 1em;
	border: 1px solid #c93;
	padding: 0.4em 0.5em;
	background-color: #ffc;
	font-size: 80%;
}
.hint .hint-pointer {
    position: absolute;
    left: -10px;
    top: 2px;
    width: 10px;
    height: 19px;
    background: url(/images/pointer.gif) left top no-repeat;
}

The mootools bonus: I can easily add a fade effect, which I did. I further simplified things by adding the hint-pointer div at render time with mootools instead of repeating the markup over and over again. A little DRY from Rubyland.

Yay.

By jasonb | Posted in JavaScript | Comments (2)

dry model validations with class_eval

8/6/2007 – 4:25 pm

I find myself having to validate an email address in a couple of different models. I've had to refine the pattern a couple of times, but don't always remember to do it in every place, which is silly anyway. Since I don't feel the need to delve into a new validation, like a validates_email_address or whatever, the following lets me simply reuse an existing validation in multiple places.

module EmailValidation
  def self.included(base)
    base.class_eval do
    validates_format_of :email,
      :with => %r{\A([-\w.\+]+)@((?:[-A-Za-z0-9]+)\.+[A-Za-z]{2,})\z},
      :message => 'appears to be invalid. Please verify it is correct.'
    end
  end
end

The above is just saved as email_validation.rb in lib/. Then later in my model.

class FooWithEmail < ActiveRecord::Base
  include EmailValidation
end

And no, an email address shouldn't have A-Z in the domain portion, but I run downcase! against that attribute before saving. Users can't really be bothered to care if a domain should be uppercase, lowercase, or mixed.

By jasonb | Posted in Rails | Comments (0)

ActiveScaffold and Mongrel death in production

7/24/2007 – 11:14 pm

Oh, I have such love for plugins that copy files around in production. I especially love it when the Rails app doesn't have permission to write to the filesystem -- why would it in production? -- and Mongrel simply goes away. Running Mongrel without backgrounding it with start-stop-daemon reveals the trickery in action.

** Starting Mongrel listening at 127.0.0.1:8101
** Starting Rails with qa environment...
/usr/lib/ruby/1.8/fileutils.rb:1246:in `initialize': Permission denied -
 /srv/rails/current/config/../public/stylesheets/active_scaffold/default/stylesheet-ie.css (Errno::EACCES)

Why?

vendor/plugins/active_scaffold/init.rb
##
## Run the install script, too, just to make sure
##
#require File.dirname(__FILE__) + '/install'

Seriously, just don't do it. At least verify the file is present before doing a blanket copy on each and every startup. Seriously.

By jasonb | Posted in Rails | Comments (1)

mootools v1.11, ajax evalScripts and Tips bugs

7/16/2007 – 11:13 pm

Mootools v1.11 as shipped has a few bugs I came across. I was assured there couldn't be any, as it's been out for a while.

[Mon Jul 16 2007] [15:36:10] 	JasonBoxatWork: Whoah, you get to box at work, Jason.
[Mon Jul 16 2007] [15:36:11] 	NICE.
[Mon Jul 16 2007] [15:36:21] 	 * JasonBoxatWork laughs
[Mon Jul 16 2007] [15:36:27] 	Try redownloading.
[Mon Jul 16 2007] [15:36:45] 	If it's still broken when you do that, it's something you're doing.
[Mon Jul 16 2007] [15:36:49] 	1.11 has been out for ages now.
[Mon Jul 16 2007] [15:36:51] 	We'd know if it was broken.

First, Firebug is encountering malformed XML errors in Content-Type: text/javascript replies. I did not have this problem with the evalScripts function in mootools v1.0, but the one in v1.11 is substanially more complicated. One new feature is it will evaluate the whole response as if you specified evalResponse yourself. The aforementioned Content-Type header triggers the behavior.

        evalScripts: function(){
                var script, scripts;
-               if (this.options.evalResponse || (/(ecma|java)script/).test(this.getHeader('Content-type'))) scripts = this.response.text;
+               if (this.options.evalResponse) scripts = this.response.text;

Above is a diff of my resolution. The JavaScript I am sending to the browser is valid and Firebug reports no errors with the above change when my returned JavaScript in script tags is evaluated by Firefox 1.5 and Opera 9.

The second issue I have encountered in mootools v1.11 is the Tips plugin will display tips partially off screen if you enable the fixed option. 

                if (el.$tmp.myTitle && el.$tmp.myTitle.length > this.options.maxTitleChars) el.$tmp.myTitle = el.$tmp.myTitle.substr(0, this.options.maxTitleChars - 1) + "…";
-               el.addEvent('mouseenter', function(event){
+               el.addEvent('mouseover', function(event){
                        this.start(el);
-                       if (!this.options.fixed) this.locate(event);
-                       else this.position(el);
-               }.bind(this));
+                       this.locate(event);
+               }.bindWithEvent(this));
                if (!this.options.fixed) el.addEvent('mousemove', this.locate.bindWithEvent(this));

Above I reverted to the mootools v1.0 way of handling the mouse event. Now with fixed enabled tooltips will appear on the side of the window where each will be fully visible.

By jasonb | Posted in JavaScript | Comments (1)

Flash Upload Progress for Rails with FancyUpload for mootools

7/15/2007 – 11:07 pm

There are a ton of possible upload progress solutions for Web applications that need them. I had been using the upload progress that came with Merb, but I had modify mup.js to make Ajax calls using mootools instead of Prototype. Ultimately the progress status wasn't updating consistently, so I sought a pure mootools compatible solution.

That brought me to the FancyUpload upload progress solution by Harald Kirschner. It's Adobe Flash based, meaning upload status can be determined on the client side using Flash itself. It's easy to use and fairly sexy, but Flash comes with its own bundle of deep hurting.

I discovered that Flash 8/Win does something extremely retarded. It sends an initial POST with a Content-Length of 0. Sane multipart MIME parsers simply toss it out as the garbage it is. Unfortunately, that means Rails via CGI.rb and Merb are raising exceptions and Flash is giving up. I don't agree with the solution to directly monkey patch CGI.rb; I don't know enough about multipart MIME parsing to feel secure running a monkey patched CGI.rb on a production system.

Moreover, I am using Merb, and its multipart MIME parser is completely different than CGI.rb, meaning I'd have to come up with my own monkey patch without potentially compromising security. No, thanks.

Instead, I am taking advantage of Apache 2.2, which I am already using for mod_proxy_balancer between several Mongrel instances, to fix Flash 8's broken HTTP POST. If you've ever wondered how to do some extremely strange stuff using Apache's powerful rewrite rules, examine my solution to Flash 8's suckiness below.

BrowserMatch ^Shockwave Flash$ swf=true
RewriteCond %{HTTP:Content-Length} ^0$
RewriteCond %{ENV:swf} ^true$
RewriteRule . %{REQUEST_URI}?flash8win=1 [QSA]

RewriteCond %{QUERY_STRING} flash8win
RewriteRule . - [E=strip_content_type:true]
RequestHeader unset Content-Type env=strip_content_type

First, we check to see if it's a Flash client. Flash 8 wasn't reporting a version string in any of my packet captures, so I simply used the string 'Shockwave Flash'. Apache's BrowserMatch allows you to set an environment variable which I do if the client is Flash.

Next, I perform a series of checks to see if the current request is a broken one created by Flash 8/Win. If the Content-Length is 0, we have a winner. The query string is overloaded with a string signaling Flash 8/Win has been detected.

Finally, I perform a match and if my query string indicates I found Flash 8/Win, I set an environment variable which RequestHeader can match upon. The Content-Type header is then removed. Without a Content-Type header indicating a multipart POST, the MIME parser is skipped and CGI.rb and Merb's parser have nothing to do and I can ignore the empty request entirely.

def handle_upload
  if ! params[:flash8win].blank?
    render :text => '', :status => 200
    return
  end
  .. other code ..
end

Clearly, you can do some amazing things with Apache2 and its rewrite rules.

It's useful to note that under Merb 0.3.4, a 500 error is returned when the MIME parser encounters something worthy of raising an exception. Flash 8/Win doesn't care if it gets a 500 response to its initial Content-Length: 0 request. It did care when it received a 404 Not Found, which is what Merb 0.2.0 and Mongrel using CGI.rb return. If I simply upgraded to Merb 0.3.4, I would never have written any rewrites rules, but had tons of mysterious 500 errors in my merb.4000.log.

Yay Flash.

By jasonb | Posted in Rails | Comments (6)

I hate Citi and my worthless Citi Professional Card

6/6/2007 – 6:21 pm

Damn. I've never quite had such a worthless experience with a company before. I shall attempt to explain. I signed up for a Citi Preferred Rewards card last year. When I called to activate the card, some dude from India harasssed me continuously about their credit production plan. But, I like signup bonuses, so I thought I'd enjoy more pain and applied this year for a Citi Professional card with 15,000 signup bonus points. Oops.

To obtain my points, I make my $0.59 purchase. I bought a bagel, I think. Anyway, at some point it shall be time to pay that. So today, I sit down to login to citicards.com. Not so fast, though. The professional card requires a new login, since it's apparently a business card.

First, Citi's Web site does not work at all in Opera. It works poorly in Firefox, with a full screen flash thing appearing and obscuring access to most of the front page. Once you login, many drop down menus are obscured by more flash crap.

Eventually, I managed to navigate to a link to signup a new card. Somehow I have a security word, but these days companies won't even tell you what the hell you security word was about. So, I made some random guesses and ultimately got locked out. Lovely.

I just want to pay this crap and be done, so I call the number on the card. My balance isn't due yet, according to the recording, so I can make no payments. I tell the voice to go play in traffic with a pony, but results in my being transferred to a real bitch.

I explain that the Web site won't go and I just want to pay off my balance. She explains it isn't due yet. I tell her how I got burned last time by Citi's Web site. It took two weeks to verify EFT and I ended up paying the last day by phone for $14.95 charge. Pissed me off, a lot. I explain I just want this to go away, to pay it. She tells me "this isn't a crisis, the statement hasn't even closed yet". From tone of her voice, I believe she meant to say 'calm the fuck down'.

She had that tone most of the conversation and seemed uninterested in helping in any fashion. She explained $5 is the minum payment for EFT or pay-and-get-screwed-by-phone, so I'd have to mail a cheque, the only way I can pay. Of course I just moved, so I have no clue if and when my mail will get sorted and if you miss a payment you risk ending up in universal default, a backdoor method by which all credit card companies can fuck you.

She couldn't help with the Web site, so transferred me to a lame technical support representative. I explained I just wanted to login to the silly site. He unlocked my card number and hinted my security word was apparently a pet's name. I had no clue; I don't really give a damn about security words. Citi's site didn't indicate if you are supposed to specify a security word you want or the security word you have when you add a new card. Who knows?

After that, he said I can just signup again and to not enter the word with any extra spaces and in all lower case and basically hung up on me.

Citi's customer service never fails to disappoint and their products are simply horrible. What a shit company. There's simply no comparison to Chase. Chase is awesome and the best card company I have ever done business with. The Web site works, everyone I have spoken with has been helpful and not overtly rude.

Next time the signup bonus comes around for a new Citi card, I think I'll just skip it. This nonsense isn't worth $150 in gift cards.

Online Payment Error Wednesday, June 6, 2007

Your ability to make online payments for this account is not active yet. It can take up to 6 business days from the setup date. To make a payment today, call the Customer Service number on the back of your card.

What a joke.

By jasonb | Posted in Rants | Comments (5)

Embarq DSL weak value!

6/4/2007 – 4:08 pm

Never in my life have I been so thoroughly ecstatic with a telecom provider. (Of course the honeymoon is over. Embarq is highly unreliable for DSL. Worse service I have ever had thus far.) Last Wednesday I ordered service completely online using the Embarq Web site. It was straightforward and easy. It was possible to order a bare phone line with no services of any kind except high speed internet (ADSL). I didn't have to fight the Web site to exclude stupid bundles -- or in the case of Verizon or Bellsouth simply give up; none were forced upon me.

The total cost out the door for monthly service for bare ADSL is only around $50, notably cheaper than my Verizon ADSL or my Bellsouth FastAccess DSL from years ago. The upload speed is also the fastest, at 576 kbits/sec as reported by my modem.

Further, although scheduled for Tuesday, Embarq arrived a day early to install service. The installation fee is $50, but is rebated online in a simple fashion. There's apparently a $10 gift cheque as well for signing up. Of course there's the usual one-time connect fee, but it was as reasonable as such a charge can be.

It's extremely rare I have such high praise for any company; especially a utility. If Embarq is an option in your area, I'd check it out. The upload speed is very good, if you're into that, and the price is quite low. If you don't need the quota'd, high download speed of Brighthouse then Embarq ought to be just fine. (And they're offering 25GB of online storage for backups -- No clue what that is but it sounds interesting if it's legit.)

Update, August 1st. It's useful to note that with Embarq so cheap, it is seriously oversubscribed. I am encountering between 50-90% packet loss on some routes when playing BF2, which makes gaming on Embarq mostly worthless. In fact, Embarq is the worst DSL provider for gaming online I have ever had. Previously, I had Verizon and before that, Bellsouth. I also had RoadRunner/Brighthouse at one point, and it was hands down the best for gaming over any DSL provider I've ever had.

I've also been unable to signup to Embarq's Web site for a month now. It fails with a HTTP 100 Continue response, which is really weird. I've tried with both Firefox and Opera to no avail. I find that disappointing, since Embarq is a broadband company!

If you want to game online, don't get Embarq!

Update, August 20th. Embarq is unreliable for connectivity, too. I've lost connection twice today, requiring hard power cycles of the modem and continuously trying to get a new DHCP lease. It's also failed twice like this in the past week besides today. It's really awful service. Embarq seems to sell Always Off Service (tm).

By jasonb | Posted in Rails | Comments (0)

Debian Etch NVidia direct rendering no

5/27/2007 – 9:50 pm

I so rarely use 3D on my Debian GNU/Linux desktop, I didn't bother to verify it was working when I upgraded to Etch. Actually, it's probably been busted since I moved to the unofficial Xorg in Sarge Backports in December. The Debian WiKi has the solution, essentially.

nx@faith:/home/.nx$ glxinfo | egrep "glx (vendor|version)"
server glx vendor string: NVIDIA Corporation
server glx version string: 1.4
client glx vendor string: SGI
client glx version string: 1.2

They both must be NVIDIA. Obviously mine was not. But why? A quick strace revealed glxinfo is still looking under the old /usr/X11R6/lib/libGL.so.1 for its client library. The solution was fixing a symlink.

faith:~# cd /usr/X11R6/lib/
faith:/usr/X11R6/lib# ls -1
libGL.so.1 -> libGL.so.1.2
faith:/usr/X11R6/lib# ln -sf /usr/lib/libGL.so.1 libGL.so.1

By jasonb | Posted in Quick Guides | Comments (1)

Firefox 2 an abject failure, Opera restored my faith

5/9/2007 – 8:40 pm

Okay. Enough. I've been using Firefox since before the 1.0 release came out. I enjoyed version 0.8, even if it crashed a little. It was light weight and fast. Browsing was fun and fully featured again. KHTML wasn't bad, but still didn't support enough of the Web as I would have liked.

I finally moved to Firefox fulltime last year, hanging onto the 1.0x series for quite a while. I found it leaked memory, despite having only flashblock installed. I'd find it was using upwards of 500M of RAM after a week of being left open, having cycled through many tabs throughout the week. Eventually, predictably, and consistently I had to restart my browser for that feeling of speed and joy to return.

I finally moved to Firefox 1.5 when I stuck Kubuntu 6.06 on my laptop. It still felt like the same Firefox, mostly. I had been mostly satisfied with 1.07. There weren't any improvements in 1.5 of much interest to me. I still had to restart 1.5 once a week, which felt wrong. I'd grown so accustomed to dealing with Firefox's quirks I stuck with it. Firefox had by this time surpassed my daily usage of Konqueror from years prior.

With Debian Etch being released and my desire to install Kubuntu 6.10 on my laptop, I found myself moving to Firefox 2.0. I was mostly happy with 1.5 -- just like 1.0x -- with the obvious exception of the weekly restarts. However, I found Firefox 2.0 actually boasted some features I enjoyed. The spell-as-you-go, which KDE's Konqueror has had since at least KDE 3.3.2, finally made it into Firefox as a shipped feature. That was pleasing. Firefox 2.0's recovery of the session after a failure was also pleasing to me.

Unfortunately, I stumbled upon several issues that are complete deal breakers. First, the memory leak issue is still presently, though in Firefox 2.0 I found myself restarting Firefox more often than ever before. My Web site choices and browsing habits have not changed between 1.5 and 2.0. Worse yet, Firefox 2.0 was completely unusable on my laptop, saping battery power with its nonstop gettimeofday() calls. Worse still, on my workstation, Firefox 2.0 would hang for five seconds between opening new, blank tabs. I have no idea why and the behavior persists even after a completely fresh start of Firefox 2.0 with no open tabs. The browser history list also seems to lag out for up to ten seconds on a completely idle system when I start to type an address. Such behavior was unheard of when I used prior releases of Firefox.

As such, I downgraded to Firefox 1.5 on both my Debian Etch workstation and my Kubuntu 6.10 laptop. However, I have finally grown bored with weekly restarts and other minor quirks with Firefox. In a fit of rage, I did something I'd decided I'd never do: I downloaded a copy of Opera, a QT-based browser that for years was only available in exchange for money amid a sea of decent, completely Free browsers. Now, the Free part doesn't much matter to me, but the free part certainly did. Opera finally removed that roadblock, but with Firefox, I decided I didn't much care at the time.

However, in the few hours I have been using Opera, I have found it to be nothing but a pleasure to use. It's been easy to configure, it's extremely fast. I quickly downloaded a nice KDE-like theme for it. I tweaked the fonts a little. I started browsing familiar pages finding they all render exactly as I expected they should. Thus far, Opera is even using a sensible quantity of RAM. (I have 1GB of RAM, but when Firefox 2.0 gets up around 512M of used RAM, that's just insane...)

Needless to say, I am very, very pleased.

Today I became an Opera user.

Update, May 23rd. Still an Opera user. Awesome browser!

By jasonb | Posted in Rants | Comments (1)

Firefox uses 100% of CPU after hibernate

4/30/2007 – 12:52 am

I recently upgraded to Firefox 2.0.3. Previously I had been using a 1.5.x version. After discovering Firefox frequently using 100% of the CPU after using it only for a short while, I investigated further. It apparently is makes tons of gettimeofday syscalls. I stumbled upon a known issue in bugzilla. Mozilla runs at ~100% cpu usage after connection is interrupted or wakeup from hibernate or standby. I thought it was some page I was visiting, but apparently not. Firefox 1.5 doesn't have this issue on my hardware platform, a Dell Inspiron 1100 running Linux 2.6.20.6 with the latest release of suspend2. I have downgraded back to 1.5, which I was happy with anyway.

Update, May 14th. Apparently Firefox 1.5 suffers from the same issue on my laptop. I have switched to Opera. I am much happier for it.

By jasonb | Posted in Rants | Comments (3)

Experts-Exchange Sucks More, Employing Obfuscation to Force Signup

4/14/2007 – 4:57 pm

Which isn't to say I find Experts Exchange dot Com not to be completely worthless, rife with mostly worthless "experts" and an obscene amount of ads. Thankfully, they've taken the first step towards pulling the plug on their worthless site by forcing you to register to view any threads that appear in Google. In Konqueror, when I hover over any answer, JavaScript is used to blur it so it is unreadable. In a more recent browser, I imagine the answers are completely hidden on page load. Exports Exchange, I raise my glass to your eventual extinction. Keep up the great work!

By jasonb | Posted in Rants | Comments (5)

Silent Hunter 4 a Failure, Colossal Disappointment

4/1/2007 – 10:46 pm

I had been looking forward to the latest release in the Silent Hunter series, Wolves of the Pacific. UBISoft's track record in the past has been very poor, but I was still somehwat hopeful SH4 would be least mildly release quality when it finally hit the stores. Sadly, such is not the case. It appears SH4 is a complete and utter failure, yet another beta game shipped by UBISoft. I cannot say I am completely surprised.

Many of the issues are hard to fathom. SH4 shipped to stores without support for the imperial system. All measurements are in meters unless you know to install the latest patch, available so soon after the game hit the shelves it screams beta release. Embarrassingly, the game does not support anti-aliasing with any graphics card, which is almost unimaginable. Further the graphics are stuck at 1024x768 with no higher resolution option available for in-game graphics at present. And, there are phantom polygons! Words cannot express my disappointment.

Aside from that, we run into gameplay issues. There's still no functional player death cam, an obvious omission. The U.S. Navy medals are all wrong, which is laughable. The game ships with no clear description of the torpedo load out, so you're left to your own devices to determine what each type is. Strangely, save games can mysteriously fail. I can't imagine why you might want to save your progress. (And UBISoft Silent Hunter titles have been having various save game issues since the ones I clearly recall from SH2!)

In fact, just to demonstrate the extent to which Silent Hunter 4 is abortion-ware, the community has setup the sh4bugs Web site to track game bugs and issues. The community has all the tools in place to perform the game beta testing UBISoft didn't bother doing in the first place. The community is so desperate for a decent subsim, they'll do almost anything.

I can completely understand that. I personally spend four months working on the Improved Convoys mod for SH3. I wanted those convoy encounters to be so good... I really did. Just like I want to believe in Silent Hunter. Truely, I do.

But SH4 is too colossal a failure. I am far too busy with other projects in my life to help fix yet another UBISoft abandonware-ware release. I really wish someone else would just buy the rights to Silent Hunter and save it!

Fortunately, there's DangerDeep, which is already impressive and someday will probably easily eclipse Silent Hunter in terms of quality.

All I ask is a release quality build, a decent medal system, a functional damage model, and the abandonment of the pointless and broken crew management paradigm. Honestly, SH2 got most of these things correct! It's amazing this is so much to ask for, especially the release quality part!

Sigh. *goes back to BF2*

By jasonb | Posted in Rants | Comments (10)

Rails Migrations and PostgreSQL COPY

3/27/2007 – 1:06 am

I admit it. I am bad. I run raw SQL in my migrations. The dataset I am working with requires it of me. It must be punished. To aid in its punishment, I found a post explaining how to get at PostgreSQL's COPY from within the Postgres Ruby driver.

For a Rails migration, there isn't much extra to do.

class PunishMe < ActiveRecord::Migration
def self.up
	c = Property.connection.raw_connection
	c.exec(%q{COPY cities (city, state_id, lat, long) FROM STDIN})
	data = File.open(File.dirname(__FILE__) + '/places_cities.pgdump') {|f| f.read }
	data.each_line {|line| c.putline(line) }
	c.endcopy
end
end #class

To obtain a suitable dataset, tab delimited by default, you may call upon psql.

$ echo 'COPY places_cities TO STDIN;' |\
  psql db > places_cities.pgdump
$head -1 places_cities.pgdump
Lowgap  Populated Place NC      36.5256863      -80.8672961
By jasonb | Posted in Rails | Comments (0)

Referrer Spam Annoys Me

3/22/2007 – 6:13 pm

I had no idea my blog was so popular. 100,000 page views in ten days. That's 10,000 views a day! What kind of crack am I selling? A quick check of the logs indicates it is merely referrer spam. I've never seen it so bad, though. Serious brute force is in play. Time to clean up.

RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule ^$ %{HTTP_REFERER} [R,L,E=nolog:1]

Most of them had no UA. That probably won't last forever. A much larger site would probably have all this lost as noise anyway.

Finally, a fix for corrupt logfiles. I am anal about logs.

for file in $(seq 22 2) ; do
  zcat /var/loggies/edseek.com_access_log.$file.gz |\
  perl -ne 'if($_ !~ /^.*"([^"]+)"\s"-"$/ || $1 =~ /edseek/) { print; }' > /tmp/edseek_real.log
  awstats.pl -config=edseek -update -LogFile=/tmp/edseek_real.log"
done

Another option would be to discard log entries where the IP only hit a single page within some period of time. As stats packages adopt nofollow, I hope this becomes less of an issue. Like, email spam, it's a perpetual arms race you can't win.

By jasonb | Posted in Rants | Comments (0)