Wednesday, April 25, 2007

Amazon S3 Storage Engine

Mark Atwood, a good friend of mine, is presenting a session on Amazon S3 Storage engine for MySQL.

Primary key has to be utf-8. Results coming back in less than 70ms. "If you build it, they will come".

A bucket is fully virtually hosted, it gives you a virtual SQL CMS for S3. You can put billion photographs.

Transactional engine? No it's not. S3 service is not transactional. There are no temporal guarantees in S3. If you start doing writes on a table and someone else starts then it may take some time for it show up. The reason is S3 is very distributed. Amazon S3 replicates it all over.

If you put no WHERE clause then you can "make money fast" for Amazon.

There are many opportunities. It keeps MD5 hash of contents of Blob. There is also ACL. S3 handles HTTPS but it isn't implemented yet in the storage engine?

There are other Amazon services that can use storage engines like replication distribution via S3 and cluster persistent storage via S3.

Where to get it? It uses MySQL 5.1 storage engine interface and is available at

He is disappointed about little feedback from community on his efforts.

1 comment:

adam said...

I looked at Mark's handout PDF. Cool stuff. The only thing that would seal the deal for me would be multicolumn support. Seems like it would be easy to append a /column-name to the end of the primary key for each column to get that data element requiring n s3 gets to get a row with n-columns.