Links for Wednesday, March 29, 2006

Links for Thursday, March 23, 2006

Hey Mom I’m on TV: Mix06 Talk Preview

As part of my series of talks at Mix06, a press crew from Microsoft interviewed me yesterday. You can see the results here. We did this in one take; they did some editing and made it come out somewhat decent. As a general rule I avoid watching or listening to myself. That’s too recursive for my taste.

If you are in Vegas for Mix06, hope to see you at the Alexa Web Search talk — tomorrow at 10:00 AM (not 10:30, like I said in the video). If you are not in Vegas, Microsoft has promised to put the videos online within 4 weeks.

S3 Rocks!

Every day, a script on Syndic8 generates a backup file. This file grows in proportion to the number of feeds in the system and is currently about 1.3 GB in length. At some point I will need to generate multiple files, but for now I can handle files of this size.

As an experiment, I uploaded a recent backup file into Amazon’s new S3 storage system using the Perl / Curl sample built by the S3 team. This sample is a simple (and very elegant) Perl script which computes the proper S3 authentication parameters and then invokes the command-line version of the Curl utililty to access S3. Since I had already created an S3 bucket, I used the following simple command to upload the file:

s3curl.pl --id=MYID --key=MYKEY --put=MYFILE.tar.bz -- http://s3.amazonaws.com/jeffbarr/backup/MYFILE.tar.bz

The entire file was uploaded in just 23 minutes, for a net speed of 1 megabyte per second. I verified that the file was present like this:

s3curl.pl --id=MYID --key=MYKEY -- http://s3.amazonaws.com/jeffbarr

Note that the arguments each start with a pair of dashes, and that a further pair of dashes is used before the final URL.

This returned an XML structure, so I copied the output of the command to my public_html directory and opened it in Firefox.

It would be very cool if S3 returned the MD5 checksum of the file. That way I could run md5sum locally, and then compare it to what S3 returned. I will talk to the development team about this in the very near future.