2007-08-21

Windows Environment Variables Trap

In Windows, you can easily define environment variables from the "My Computer" icon on your desktop. It's convenient but you can't define the evaluation order, so Windows does it for you but it does it wrongly (at least on Windows XP).

Define these variables:

V1=1
V2=%V1%-2
V3=%V2%-3
V4=4-%V5%
V5=5-%V6%
V6=6

Here is what you get:

V1=1
V2=1-2
V3=1-2-3
V4=4-%V5%
V5=5-6
V6=6


So, V4 was not correctly expanded.
We can also see that the evaluation is partly based on alphabetical order (V1, V2, V3) but that there were 2 passes as V5 is correct.

So, if you want to get safe set an evaluation order on your variables by setting them in your autoexec.bat file.

2007-08-15

My First Ruby on Rails project

It's been a while since I wanted to refactor my PHP+Smarty web site and also to implement additional database driven features. The main purpose of this application is to manage table tennis tournament results.

CakePHP

Being hosted on a PHP box, my first idea was to look for Ruby on Rails clone in PHP.
I found CakePHP to be the only serious one able to run on PHP4 so the choice was easy.
I spent few hours during 3 weeks to learn the framework and start my project, there was no book to help so I relied mostly on simple tutorials and users forums.
CakePHP is a good framework with may useful features (some of them are even more advanced than Ruby on Rails equivalent ones) and a great community but documentation is sparse and worse: it suffers from PHP's poor syntax (at least from a Java developer's standpoint).
After 3 weeks, I decided to stop and switched to Ruby on Rails even if it meant switching to a new web hosting company.

Switching to Ruby on Rails

I bought the "Agile Web Development with Rails" book and read several tutorials.

Using existing database

First, I created my application using rails ipttc then I created a migration from my existing database.


ruby script/generate migration Initial
rake db:schema:dump


Then I copy/pasted from schema.rb into the up method of db/migrate/001_initial.rb

rake db:migrate

I found that my legacy schema was not following some of Rails conventions like having a numerical id for tables, I decided to keep it simple and adapt the rails conventions, maybe I'll dig further if needed.

Being a lazy typist, I looked at ways to generate my models from existing database and I found Dr Nic's Magic Model Generator which created 7 models with validation rules in a snap, Thanks Doc! By reviewing the generated models, I was able to learn quickly about validations and relations and I really discovered something unexpected: it did create some kind of indirect relations: "has many through".

Tournament has many Medals.
Player has many Medals.
Tournament has many Players through Medals.

And it does make a lot of sense, it's like saying:

Tournament has many Medalists.

After that, I started to play with scaffolding and I was a bit disappointed because it did not take into account relations between models. I searched for better scaffolding and found ActiveScaffold and Hobo. ActiveScaffold provides dynamic scaffolding using Ajax and nice presentation. Hobo is more a framework on top of Rails and provide additional nice features like a security model with users, roles and permissions. Both of them are very promising but they provide only dynamic scaffolding while I am more looking for a code generator to be able to learn and modify things for my needs. Also, I'm still too new in Rails to be able to select an additional framework, I must try to understand the standard Rails features first.

Fortunately, I was also looking at RESTful implementation in Rails and found that there was a new scaffold code generator that was deprecating the original one.
The good thing about scaffold_resource is that it lets you to generate the model, the views, the controller and the database table in migration at once.
You just have to list the name of your fields and types, for instance for Player and Country:


ruby script/generate scaffold_resource Player full_name:string first_name:string last_name:string birth_date:date country_id:integer gender:string class:integer dead:boolean eligible:boolean

ruby script/generate scaffold_resource Country code:string name:string

rake db:migrate


Conclusion

My application will be RESTful and I will create a new database schema.

2007-08-12

Code Red Alert

A little story that happened to me as a hobbyist webmaster of www.ipttc.org

In previous months, I have seen our network bandwidth consumption increasing, 2 months ago I found that there was a lot of traffic coming from servers in Vietnam where our sport is not very developed but at this was at the end of the month I did not pay too much attention.

On June, 13th, I noted a major traffic increase, I looked at the most downloaded pages and was surprised to find audio files (.wma) while we don't distribute music of course.

Then, I found that there were about 1500 such files in one directory which was the upload directory of forum for attachments. So, everything indicated that a hacker did exploit a vulnerability in the file upload module of the forum.
I decided to remove the files but it was not possible, so I renamed the directory and I logged a support ticket to get administrators to do the job.
I un-installed the file upload module and updated the forum software to most recent version.

I sent a mail to my committee qnnouncing that our web site would probably get unavailable because our network bandwidth for the month could be exceeded within next hours or days and that the downtime could last until end of the month.

Next morning, I got a mail from support saying that they deleted the files.

Unfortunately, our bandwidth got exceeded during the night and so our site was down.

I decided to look carefully at the web server log files and I found that all requests for music files came from one site www.muzic9.com. In fact, this site proposes "free" music, you choose an album and then click on a song, it then redirects transparently to an external site. So it means that when clicking on some songs you actually downloaded them from www.ipttc.org! I sent an email to the webmaster asking him to delete all links pointing to our site get and that he no longer accepts such links.

I did a "whois" query to find the site owner:


TUAN
TUAN TUAN (tuan.maxviet@gmail.com)
1.8633630
Fax: 1.8633630
Some where in VN
address
HCM, HCM 70000
VN


So the owner was from Vietnam, same country that consumed our bandwidth last month.
I did send same email to this address. No need to say, I never got any reply.

I put some additional protections in place and now monitor more seriously my bandwidth consumption report and web log files.

2007-08-10

Subversion quick protocol benchmark

Subversion lets you choose the protocol used to connect from client to server, it has an impact on the authentication scheme but also on data transfer rate.

I ran a checkout on 2500 files and 50 MB with client running locally on server. As expected, "svn" was the fastest: "http" was 66% slower and "https" was 135% slower.

Also an export of same source tree is much faster than a checkout (up to 3 times faster in my tests depending on the number of files) because svn client does not need to create a local copy of each file nor metadata files (checksums, logs, ...) in .svn directories. So, it could be a little optimization for a build that does not need to commit any change.

Of course, when the client is remote, the impact of a slow network connection tends to lower these differences.